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Executive  Summary 

Spatial  disorientation  (SD)  is  a  normal  human  response  to  accelerations  in  flight  (Previc, 
1998),  and  has  existed  since  early  flight  (Ocker  &  Crane,  1932;  Macurdy,  1934).  Its  cost  to  the 
US  military  is  over  $300  million  per  year  (Heinle,  2001;  Davenport,  2000),  with  comparable 
costs  to  US  civil  aviation  (Veronneau,  2000).  Despite  significantly  increased  research  over  the 
past  decade  (Previc  &  Ercoline,  2001),  the  rate  of  accidents  caused  by  SD  has  not  decreased 
(Ercoline  et  al.,  1994a).  While  the  most  recent  research  emphases  have  been  on  understanding 
the  physiology  of  SD,  the  translation  of  the  new  knowledge  into  tools  (e.g.,  training,  displays, 
automation)  that  help  pilots  avoid  SD  and  minimize  its  effects  if  it  does  occur,  has  not  occurred. 

The  goals  of  the  research  reported  here  were  to  apply  the  recent  research  results  to  practi¬ 
cal  methods  that  enhance  pilot  attitude  awareness  and  combat  SD.  The  vast  majority  of  SD- 
induced  accidents  are  from  Type  1  (unrecognized)  SD  (Ercoline  &  Previc,  2001).  The  major 
challenge,  then,  is  to  convert  Type  1  SD  events  into  Type  2  (recognized)  events  so  that  the  pilot 
can  recover.  A  solution,  even  a  partial  one,  to  Type  1  SD  events  will  greatly  improve  the 
appalling  SD  accident  statistics. 

Considerable  progress  has  been  made  in  providing  pilots  with  countermeasures  to  SD.  It 
is  the  premise  of  this  Phase  I  effort  that  more  effective  multisensory  countermeasures,  applied  in 
an  intelligent  fashion,  are  needed  and  possible,  and  that  modeling  and  simulation  are  the  most 
cost  effective  means  to  identify  and  test  countermeasures  for  different  aircraft  under  different 
flight  conditions. 

There  are  8  chapters  in  this  report.  The  first  chapter  describes  the  impact  and  nature  of 
spatial  disorientation  in  aviation.  The  chapter  gives  a  brief  background  on  the  neuro-physio- 
logical  processes  that  underlie  SD.  The  3'^'^  chapter  describes  multisensory  cockpit  technologies 
that  could  reduce  the  quantity  and  severity  of  SD  mishaps.  The  chapter  presents  our  innova¬ 
tive  theoretical  and  practical  approaches  to  modeling  pilot  attitude  awareness  and  SD.  Our 
approach  demonstrates  how  human  performance  modeling  can  be  used  to  study  the  effects  of  SD 
reduction  from  new  and  future  cockpit  technologies,  and  thus  to  reduce  the  time  and  costs 
needed  to  test  prototypes  and  to  push  more  promising  technologies  into  flight  test  sooner  than 
with  a  more  traditional  approach.  Modeling  also  suggests  effective  training  strategies  and 
scenarios. 

Realistic  Air  Force  scenarios  are  in  Chapter  V.  These  mission  scenarios  put  the  pilot  at 
considerable  risk  of  disorientation  and  serve  as  the  foundation  for  our  SD  Aiding  System  proto¬ 
type’s  testing  and  demonstration.  The  scenarios  include  a  slow-onset  Leans  event,  which  is  the 
most  common  SD  event,  as  well  as  a  quick-onset  Coriolis  event,  which  is  one  of  the  most  com¬ 
pelling  and  lethal  types  of  disorientation.  Our  assertion  is  that  if  we  can  credibly  model  these 
types  of  disorientation  events  and  aid  in  their  recovery,  our  approach  will  be  extendable  to  other 
SD  illusions.  (And,  in  fact,  when  we  received  actual  flight  data  from  the  Air  Force  Safety 
Center,  that  data  set  ran  reasonably  well  through  oiu  SD  Aiding  System,  proving  its  robustness.) 

The  6*  chapter  provides  the  rationale  and  design  for  an  intelligent  aiding  system  that 
helps  with  SD  event  recoveries.  The  7*’’  chapter  highlights  our  major  accomplishments  in  Phase 
I.  Chapter  VIII  discusses  Phase  11  goals  and  plans.  The  emphases  in  Phase  11  will  be  to  validate 
our  models  via  hmnan-in-the-loop  testing,  and  then  to  evaluate  our  model-based  countermea- 
sxires  via  pilot-in-the-loop  experiments. 
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I.  Introduction 

spatial  disorientation  (SD'),  simply  referred  to  as  vertigo  until  becoming  the  subject  of  scientific 
research,  is  a  normal  response  to  neuro-physiologically  confusing  stimuli  in  the  flight  environ¬ 
ment.  The  human  body  developed  over  eons  to  perceive  motion  on  land  in  relation  to  the  surface 
of  the  earth.  In  flight,  these  physiological  systems  can  give  the  brain  erroneous  orientation  infor¬ 
mation.  Everyone  who  flies  will  eventually  experience  some  of  these  phenomena;  some  pilots 
will  experience  them  more  acutely  than  others. 

An  increased  reliance  on  air  power,  the  current  USAF  advantage  in  night  operations,  and  the 
emphasis  on  reduced  crew  numbers  will  accelerate  the  already  increasing  incidence  of  tragedies 
that  result  fi'om  difficulties  of  the  human  sensory  system  to  spatially  orient  in  the  3D  world  of 
flight.  In  order  to  advance  our  mastery  of  the  flight  environment,  a  thorough  understanding  of 
disorientation  phenomena  and  appropriate  countermeasures  must  be  devised.  The  focus  of  this 
report  will  be  to  recommend  a  modeling  and  simulation  approach  that  will  be  based  in  human 
neuro-physiology,  flight  dynamics,  and  physics,  which  will  enable  the  rapid  and  economical 
prototyping  of  disorientation  countermeasures. 

This  conflict  between  one’s  senses  and  reality  has  received  much  recent  research  attention.^ 
Understanding  the  physiological  causes  of  SD  is  essential  to  solving  the  problem,  and  will  be 
discussed  in  Chapter  II  of  this  report.  A  next  step  is  to  quantify  the  contributory  values  or  ranges 
for  the  various  physiological  mechanisms.  For  example,  how  much  do  optical  illusions  contrib¬ 
ute  to  SD-related  accidents?  How  effective  is  a  particular  attitude  display  in  helping  to  maintain 
SO?  Some  answers  are  beginning  to  emerge.  For  example,  Gomez  (2002)  asserts,  “90%  of  all 
inputs  to  the  brain  that  give  the  sense  of  orientation  are  visually  acquired.”  This  assertion 
implies  that  SO  displays  should  be  visual,  as  opposed  to  auditory  or  tactile  displays,  which,  pre¬ 
sumably,  contribute  only  10%  of  the  orientation  inputs  to  the  brain.  However,  the  SD  problem  is 
too  important  to  rely  on  only  one  sensory  channel.  Also,  because  SD  is  often  due  to  a  sensory 
conflict  or  distraction,  using  only  one  sense  (vision)  to  combat  it  is  ill  advised.  A  multisensory 
approach  is  needed. 

Terminology 

Several  definitions  have  been  proposed  for  SD  to  help  guide  research  and  a  generally  accepted 
working  definition  has  emerged  (Ercoline  et  al.,  1994b).  A  pioneer  in  the  field  of  SD  provided 
the  following  operational  definition  of  SD  as  “an  erroneous  sense  of  the  magnitude  or  direction 
of  any  of  the  aircraft  control  and  performance  flight  parameters”  (Gillingham,  1992). 

For  the  purposes  of  this  report,  we  consider  SD  in  layman’s  terms  as  a  loss  of  attitude  awareness, 
and  not  as  a  loss  of  geographical  awareness.  Attitude  awareness  is  the  pilot’s  active  knowledge 
of  the  aircraft’s  pitch,  roll  and  yaw  (particularly  the  first  two),  and  the  likely  near-future  values 
of  those  factors  that  comprise  aircraft  attitude.  We  place  attitude  awareness  (and  spatial  disori¬ 
entation)  in  the  broader  context  of  situation  awareness  and  spatial  orientation  as  follows; 


'  All  acronyms  are  defined  in  the  last  chapter  of  this  report. 

^  For  example,  the  Spatial  Disorientation  Symposium  (Nov  2000)  in  San  Antonio,  TX  had  die  following  relevant 
presentations:  Gallimore,  J.J.  &  Liggett,  K.K.  Implications  of  spatial  sensory  reflex  research  on  primary  flight 
display  design  and  Cheung,  R.  Non-visual  spatial  orientation  mechanisms.  See  also  the  web  site; 
http://www.spatiald.wpafb.af.mil/library_san.asp. 
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Situation  awareness,  the  broadest  concept,  defines  the  pilot’s  understanding  of  the  current  and 
future  situation  pertaining  to  any  of  a  number  of  dynamic  variables  in  the  cockpit,  such  as  the 
status  of  automation,  task  completion,  or  the  spatial  behavior  of  the  aircraft.  The  latter  form  of 
situation  awareness  defines  spatial  awareness  or  spatial  orientation.  Spatial  orientation  in 
turn,  pertains  to  knowledge  of  three  types: 

i.  Where  am  I  (location,  altitude)? 

ii.  Which  way  am  I  going  (velocity  vector:  trajectory  and  vertical  velocity)? 

The  answers  to  questions  (i)  and  (ii)  define  geographical  awareness. 

iii.  Which  way  is  up  (pitch  and  bank)? 

The  answer  to  question  (iii)  therefore  defines  attitude  awareness. 

Thus  spatial  disorientation  refers  to  the  loss  of  attitude  awareness.  As  we  will  see,  this  loss  is 
caused  by  degraded  visual  information  regarding  the  true  horizon,  plus  the  input  of  misleading 
vestibular  signals. 

There  are  presently  three  types  of  SD,  as  identified  by  researchers  (Ercoline  &  Previc,  2001): 

•  Type  1,  unrecognized,  SD:  Refers  to  the  conditions  that  occur  when  a  pilot  is  not  aware 
of  critical  control  or  flight  parameters  of  the  aircraft.  The  aircraft  is  controlled,  therefore, 
v/ith  erroneous  assumptions  of  the  aircraft  attitude.  Type  I  SD  is  generally  considered  to 
be  the  most,  deadly  since  the  pilot  is  not  aware  that  anything  is  wrong  (Brown  et  ah, 
2000).  It  is  the  most  common  type,  usually  associated  with  sub-threshold  roll  or  pitch 
motion,  and  it  may  be  due  to  distraction  or  target  fixation,  such  that  the  pilot  does  not  pay 
sufficient  attention  to  the  aircraft  attitude.  Especially  in  civil  aviation,  SD  results  fi-om 
relatively  inexperienced  pilots  flying  into  poor  or  marginal  weather,  losing  their  normal 
visual  attitude  cues,  having  a  trigger  event  (e.g.,  a  distraction  or  moving  the  head  off-axis, 
such  as  to  look  for  and  pick-up  a  chart),  and  “letting”  the  aircraft  slip  into  an  undesirable 
attitude  (Veronneau,  2000). 

•  Type  2,  recognized,  SD:  Associated  with  known  conflicts  between  2  or  more  orientation 
senses,  such  as  degraded  vision  and  otolith  stimulation.  Type  2  SD  includes  those  mis¬ 
haps  in  which  there  is  a  conscious  conflict  between  what  the  pilot  is  sensing  and  what  the 
instruments  indicate  the  airplane  is  doing.  In  this  type  of  SD,  pilots  typically  believe  that 
the  aircraft  instruments  are  malfunctioning;  they  may  not  believe  that  they  are  disori¬ 
ented,  but  they  know  that  something  is  wrong.  Training  typically  focuses  on  what  to  do: 
believe  the  instruments,  fly  a  smooth  recovery,  and  exit  the  conditions  that  led  to  SD  as 
soon  as  possible. 

•  Type  3,  incapacitating,  SD:  Typically,  the  pilot  is  aware  of  losing  SO,  is  trying  to 
correct  the  problem,  but  is  unable  to  provide  the  necessary  aircraft  inputs.  There  are 
often  fatal  results  due  to  a  complete  loss  of  aircraft  control. 

Some  disorienting  illusions  fit  nicely  and  uniquely  into  the  type  categories  whereas  some  transi¬ 
tion  from  one  type  to  another,  such  as  when  a  pilot  initially  does  not  recognize  an  SD  event,  but 
then  does  detect  it.  Early  pilots  discovered  all  of  the  illusions  that  we  know  about  today;  only 
the  names  and  physiological  explanations  have  changed.  Unfortunately,  the  frequency  of  inci¬ 
dents  has  increas^  as  more  ph;^iological  demands  are  placed  on  the  pilots  of  high  performance 
aircraft. 
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Spatial  disorientation  and  loss  of  geographical  awareness  are  distinct,  and  lead  to  different 
classes  of  accidents.  Loss  of  geographical  orientation  is  a  major  cause  of  controlled  flight  into 
terrain  (CFIT)  accidents.  Here  the  aircraft  is  clearly  under  effective  pilot  control.  In  contrast, 

SD  (loss  of  attitude  awareness)  involves  an  explicitly  uncontrolled  aircraft,  leading  to  the  class  of 
SD  accidents. 

While  loss  of  geographical  awareness  and  spatial  disorientation  (SD)  are  thereby  distinct  due  to 
their  association  with  geographical  awareness  and  attitude  awareness,  respectively,  the  two  states 
are  nevertheless  closely  linked.  First,  any  unnoticed  bank  or  pitch  (Type  1  SD)  will  typically 
lead  to  a  loss  of  altitude,  and  therefore  to  trajectories  toward  the  terrain.  Second,  any  noticed 
attitude  changes  (Type  2  SD)  can  easily  lead  to  a  diversion  of  attention  away  from  navigation 
and  maintaining  geographical  awareness,  as  the  pilot  places  top  priority  on  recovering  attitude. 

When  phrases  such  as  loss  of  situational  awareness  (LSA)  and  CFIT  are  used  in  the  aerospace 
literature  concerning  experimental  research  and  accident  investigation,  care  should  be  taken  to 
use  them  in  ways  that  will  clarify  the  issues  and  avoid  confusion  with  SD.  Unfortunately,  the 
research  areas  for  LSA  and  CFIT  are  not  as  mature  as  the  SD  research.  It  is  also  clear  that  SD 
researchers  do  not  often  consider  the  wealth  of  ideas  in  the  LSA  literature;  nor  do  the  S A  inves¬ 
tigators  seem  to  be  acquainted  with  the  SD  literature  as  well  as  might  be  expected.  As  Lawson  et 
al.  (2002)  point  out,  the  two  distinct  fields  of  research  could  benefit  fi’om  a  better  acquaintance  as 
they  have  much  in  common. 

Magnitude  of  the  Problem 

The  seriousness  of  SD  phenomena  to  the  flight  environment  cannot  be  pverstated.  Of  323  of  the 
most  serious  (Class  A^)  mishaps  in  the  U.S.  Air  Force  from  1991-2000, 20.2%  were  SD  related 
(Davenport,  2000).  Further,  it  is  estimated  that  89%  of  general  aviation  SD  events  are  fatal 
(Nall,  1999).  Survivors  are  typically  non-instrument  rated  pilots  suddenly  in  awful  meteorologi¬ 
cal  conditions  trying  to  complete  their  flights  and  often  are  fortunate  just  to  survive.  Holmes  et 
al.  (2003)  have  estimated  that  roughly  80%  of  pilots  have  directly  experienced  SD  incidents, 
although  these  typically  do  not  lead  to  mishaps.  SD  mishaps  are  often  fatal,  accounting  for  the 
loss  of  about  40  lives  per  year  in  the  U.S.  Air  Force,  Navy  and  Army  combined  (Braithwaite  et 
al.,  1998;  McGrath,  2000). 

Estimates  vary  widely  concerning  the  prevalence  of  SD-related  mishaps  (Lyons  et  al.,  1994). 
Gillingham  (1992),  Lyons  et  al.  (1994)  and  Johnson  (2000)  believed  that  most  published  SD 
estimates  were  too  conservative.  Gillingham  (1992)  pointed  out  problems  with  mishap  classifi¬ 
cation  and  conjectured  that  SD  caused  as  many  as  2-3  times  more  U.S.  Air  Force  mishaps  than 
the  reported  statistics  indicate.  Johnson  (2000)  stated  that  SD  mishaps  in  the  U.S.  Navy  maybe 
twice  as  likely  as  the  statistics  portray,  accounting  for  26%  of  all  Class  A  mishaps  (US  Navy, 
1991),  and  claiming  nearly  three  times  more  lives  than  non-SD  mishaps. 

Lyons  et  al.  (1994)  discussed  several  classification  problems  they  observed  in  USAF  mishap 
data,  including  failure  to  clarify  the  relationship  of  SD  to  mishaps  that  involved  continuing  visual 
flight  into  adverse  weather.  Lyons  et  al.  (1994)  noted  that  a  simple  change  in  the  accident 
reporting  form  increased  the  rate  of  categorization  of  accidents  as  SD-related.  Whereas  the  old 

^  Class  A  mishaps  result  in  loss  of  life,  or  at  least  a  million  dollars  in  damage,  or  the  destruction  of  an  aircraft. 


3 


F33615-03-M-6360 


Multisensory  Integration  for  Pilot  Spatial  Orientation 


accident  investigation  form  listed  “visual  illusions”  and  “disorientation/vertigo”  as  choices,  the 
new  form  substituted  the  currently  accepted  types  of  SD  as  possible  choices  on  the  form.  When 
the  three  types  of  SD  were  listed  on  the  form,  the  result  of  this  change  was  an  increase  in  choos¬ 
ing  SD  as  a  causal  factor  during  low-level  navigation,  from  7%  with  the  old  form  (FY86-FY89) 
to  67%  with  the  new  form  (FY90-91).  One  of  the  key  advantages  of  the  newer  form  was  that  it 
listed  “unrecognized  SD”  (Type  1),  This  change  implicitly  promoted  the  selection  of  SD  as  a 
mishap  contributor  in  cases  where  the  pilot  did  not  report  having  suffered  an  acute  and  recogniz¬ 
able  vestibular  illusion,  such  as  an  attack  of  the  Leans,  but  the  accelerations  the  pilot  experienced 
and  the  control  inputs  the  pilot  made  indicated  that  the  pilot  was  spatially  disoriented  (Benson, 
1988). 

Gillingham  (1992)  summarized  633  of  the  Class  A  aircraft  mishaps  in  the  U.S.  Air  Force  from 
1980-1989  (Table  1).  During  this  period,  356  mishaps  were  “operations  related,”  81  mishaps 
were  classified  as  “SD-related”  and  263  as  “LSA-related.”  However,  there  were  270  mishaps 
where  SD  and  LSA  were  both  mentioned  ^  contributing  factors. 

Table  1.  1980-1989  USAF  Class  A  aircraft  mishaps  -  as  categorized  by  Safety  Investigation  Boards 
_ _ (Gillingham,  1992). _ 


J 

Total 

%  Operations 
Related 

633 

56% 

13% 

Costs 

$4.45B 

$2.56B 

$539M 

Fatalities 

795 

65% 

15% 

Cheung  and  colleagues  analyzed  SD-related  accidents  in  the  Canadian  forces  (Cheung  et  al., 
1995).  They  collected  154  accident  reports  across  Categories'*  A,  B,  and  C.  They  foimd  that 
14/62  (23%)  of  Categoiy  A  accidents  had  SD  as  a  possible  causal  factor  during  the  period  1982- 
1992;  SD  was  unrecognized  by  the  aviator  in  all  but  two  of  the  14  accidents,  and  two  of  the 
accidents  appeared  to  be  of  vestibular  origin. 

Davenport  (2000)  considered  the  next  generation  of  USAF  accident  investigations  and  compiled 
SD  data  during  1991-2000  as  shown  in  Table  2.  His  report  compares  the  incidence  and  severity 
of  SD  with  another  tragic  source  of  mishaps:  loss  of  consciousness  induced  by  excessive  G 
forces  on  the  pilot  (known  as  G-LOC).  While  the  number  of  mishaps  is  decreasing,  the  number 
of  SD  incidents  as  a  percentage  of  the  total  is  clearly  increasing,  as  seen  in  these  two  tables. 

Table  2.  Comparison  of  SD  to  total  Class  A  mishaps  and  G-LOC  from  1991-2000  (adapted  from 
_ Davenport,  2000). _ _ 


Total 

%  G-LOC 
Related 

%SD 

Related 

Mishaps 

323 

3.5% 

20.2% 

Costs 

$5.5B 

$174M 

$1.4B 

Fatalities 

310 

2.5% 

19.4% 

'*  In  Canadian  terminology,  “Category’’  is  used  instead  of  “Class.” 
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The  inability  of  pilots  to  accurately  and  intuitively  perceive  aircraft  position  without  reliance 
upon  visual  cues  (firom  flight  instruments  or  the  outside  world)  is  a  major  crux  of  the  aviation 
mishap  problem.  Maintaining  spatial  orientation  cannot  be  done  in  present-day  flight  operations 
unless  one  is  attending  to  the  appropriate  visual  cues.  Unfortunately,  many  of  an  aviator’s  dis¬ 
tracting  secondary  flight  tasks  are  also  of  a  visual  nature  so  continuous  attention  to  one’s  spatial 
orientation  cannot  be  maintained  using  current  visual  displays.  Furthermore,  attention  can  also 
be  diverted  to  non-visual  sources  as  well  (Wickens,  2002),  for  example  the  distraction  of  voice 
communications  or  problem  solving.  Holmes  et  al.  (2003)  identify  “distraction/task  saturation” 
as  the  fifth  most  important  factor  in  their  list  of  causes  of  spatial  disorientation.  The  problem 
concerning  the  allocation  of  limited  attentional  resources  is  compounded  by  the  fact  that  atten- 
tional  resources  will  be  drawn  to  more  natural  and  salient  body  (vestibular)  cues  concerning  ori¬ 
entation,  which  in  the  environment  of  flight  are  not  veridical.  In  other  words,  the  problem  of  SD 
in  flight  is  not  caused  merely  by  attentional  limitations;  rather,  the  problem  is  the  formation  of  an 
incorrect,  yet  persuasive,  subconscious  tendency  to  rely  upon  vestibular  orientation  cues. 

A  tendency  not  to  recognize  hazards  during  moments  of  distraction  has  been  noted  by  SD 
researchers.  The  typical  SD  mishap  occurs  when  visual  attention  is  distracted  fi*om  the  aircraft’s 
orientation  instruments  and  the  horizon  is  not  visible  or  not  being  monitored  (McGrath,  2000). 
The  majority  of  SD  mishaps  are  related  to  Type  1  or  unrecognized  SD.  Specifically,  100%  of 
U.S.  Air  Force  SD  mishaps  during  1990-91  were  categorized  as  Type  1  SD  (Lyons  et  al.,  1993). 
In  contrast,  relatively  few  SD  mishaps  were  attributed  to  SD  that  was  recognized  (Type  2)  or  SD 
that  was  completely  incapacitating  (Type  3). 

In  years  past,  the  aerospace  community  came  to  realize  that  “pilot  error”  was  not  a  sufficiently 
specific  description  for  the  cause  of  a  plane  crash.  Now  we  must  avoid  the  temptation  to  con¬ 
sider  our  job  complete  when  we  have  identified  some  general  human  psychological  state  as  the 
cause  of  an  accident.  As  Lyons  et  al.  (1994,  page  152)  have  said  coriceming  SD  mishaps:  “. .  .if 
both  an  attention  deficit  and  SD  are  part  of  the  chain  of  events  leading  to  an  accident,  each 
should  be  separately  identified  as  a  causal  factor  if  elimination  of  either  would  have  prevented 
the  accident.”  This  advice  makes  very  good  sense  in  the  short  term.  In  the  long  term,  what  is 
needed  is  to  incorporate  separately  tabulated  factors  into  a  model  that  can  accurately  predict  the 
amount  of  variance  accounted  for  by  each  factor  contributing  to  human  error  in  flight.  This 
requires  an  initial  commitment  to  differentiate  factors  that  are  usually  aggregated  a  priori. 

In  a  retrospective  survey  of  SD-related  USAF  flight  accidents,  Davenport  (2000)  found  that 
between  1991-2000  fighter-attack  aircraft  had  the  greatest  incidence  of  SD.  The  overall  inci¬ 
dence  of  SD  in  the  USAF  per  100,000  hours  of  flight  averaged  about  0.31  during  this  time; 
whereas  for  fighter-attack  aircraft  the  rate  averaged  about  0.58. 

Summary 

Accordingly,  for  our  Phase  I  effort,  the  F- 15  and  the  F-22  are  the  platforms  of  choice.  They  are 
the  USAF’s  primary  fighter-attack  aircraft.  Their  speed,  agility,  and  terrain  following  capabili¬ 
ties  make  them  good  candidates  for  inducing  proformd  SD.  Our  Phase  I  modeling,  simulation, 
and  countermeasures  aiding  efforts  enable  progress  on  other  illusions  and  aircraft  during  our 
Phase  II  work,  or  in  work  by  other  researchers  who  can  use  our  approach  as  a  quick  and  effective 
means  to  evaluate  concepts  without  the  costs  and  risks  of  flight  tests. 
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II.  Physiology 

Major  Ocker  and  Lt  Crane. ..pulled  ...tobacco  pouches  over  the  heads  of  the  pigeons  as  a  blind¬ 
fold  and  tossed  them  out  of  the  cockpit  of  their  biplane.  They  discovered  that  the  pigeons  first 
flew  around  aimlessly  in  spirals  but  eventually  spread  their  wings  into  a  dihedral  shape  and 
floated  safely  to  earth.  This  elated  the  investigators  with  the  hope  that  humans  also  could 
navigate  ‘blind.  ’ 

Adapted  from  Ercoline,  1997 

Ocker  and  Crane  were  encouraged  to  discover  that  if  birds  did  not  panic  and  crash  to  their  deaths 
when  deprived  of  vision  during  flight  but  instead  safely  glided  to  the  earth,  then  perhaps  humans 
could  learn  to  rely  on  other  senses  to  land  safely  as  well.  Had  they  realized  that  birds  evolved  in 
the  3D  world  of  flight,  whereas  humans  evolved  from  ground  dwellers,  they  might  not  have  felt 
so  enthused.  Birds  have  had  eons  to  evolve  in  the  3D  world  of  flight  motions;  humans  have  not. 
The  observations  and  subsequent  tests  of  Ocker  and  Crane  during  the  second  decade  of  human 
flight  led  to  the  development  of  radimentary  turn  and  bank  instmmentation  for  non-visual  flying. 
Soon  thereafter,  Jimmy  Doolittle  demonstrated  blind  solo  flight,  from  take-off  to  landing,  and  the 
Army  Air  Corps  made  instmment  flying  a  required  experience  of  novice  pilots.  Students  in  the 
1930s  graduated  from  this  training  with  a  certificate  from  the  unofficial  “institute  of  the  blind” 
(Ercoline,  1997). 

Overview 

This  chapter  describes  the  physiological  difficulties  that  human  sensory  systems,  which  evolved 
in  an  earth-bound  space,  experience  in  the  aerospace  environment.  Although  SD  responses 
might  be  considered  as  due  to  the  inadequacies  of  human  sensory  abilities,  they  are,  nonetheless, 
normal  responses  to  the  3D  fli^t  environment.  This  chapter  also  describes  the  motion  and  grav¬ 
ity  sensing  systems  of  humans  -  primarily  the  visual,  vestibular,  and  proprioceptive  senses  -  and 
evaluates  how  these  senses  can  create  the  illusions  that  occur  in  flight. 

Neurophysiological  Bases 

All  organisms  Uve  in  a  3D  and  gravitational  world.  Sensory,  neural,  and  motor  mechamsms  have 
evolved  in  individual  species  for  the  representetion,  coding,  and  control  of  spatial  behavior. 

Because  we  are  constantly  moving  in  our  environmait,  we  need  accurate  information  about  the 
motion  of  objects  relative  to  our  own  position  or  motion.  When  human  motion  includes  the  third 
dimension  of  altihide,  we  can  encounter  problems  wifli  spatial  disoriaitation,  because  our  species 
evolved  on  the  ground  (or  swinging  from  trees),  where  vertical  orientation  was  not  in  doubt. 

Of  all  the  human  senses  involved  in  spatio-temporal  orientation,  vision  is  the  most  important, 
particularly  during  flight.  Even  for  Visual  Fhght  Rules  (VFR)  traimng,  private  pilots  are  rou¬ 
tinely  trained  in  how  misleading  their  reliance  upon  vestibular  or  proprioceptive  cues  alone  can 
be  during  flight.  Such  fli^t  training  “under  the  hood”  compellingly  illustrates  SD  susceptibility. 

The  vestibular  system  provides  secondary  cues  to  the  visual  system  in  contributing  to  our  aware- 
n^s  of  linear  and  angular  acceleration  and  in  helping  the  eyes  to  focus  on  an  object.  The  pro¬ 
prioceptive  system  is  an  integrated  network  of  neural  sensors  in  the  skin,  and  in  skeletal  joints 
that  provides  cues  to  tiie  vestibular  and  visual  s;^tems  about  limb  movements  in  space.  The 
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other  senses  are  only  peripherally  involved  in  a  human’s  sensation  of  orientation  and  motion,  if 
at  all,  and  will  not  be  considered  in  the  following  discussion. 

Visual  System 

The  eyes,  the  most  important  source  of  motion  information  to  the  human  brain,  send  pictures  to 
the  pilot’s  brain  about  the  aircraft’s  position,  velocity,  and  attitude  relative  to  the  groimd.  The 
overwhelming  belief  in  visual  cues  over  those  from  other  senses,  when  there  are  sufficient  visual 
cues  is  called  visual  dominance.  Visual  cues  are  more  than  adequate  for  spatial  orientation  on 
clear  days  in  VFR  conditions  with  a  well-defined  horizon.  But,  when  visibility  is  poor  (e.g., 
during  night  fl)dng,  or  in  IMC),  pilots  can  experience  SD  illusions.  Even  on  a  clear  VFR  day, 
the  eyes  can  “play  tricks”  (e.g.,  runway  and  approach  illusions). 

Vision  is  most  acute  at  the  macula  located  in  the  foveal  region  of  the  retina  in  the  back  of  the 
eye.  Figure  1  shows  the  concentration  of  rods  and  cones,  the  structures  that  convert  ambient 
light  energy  into  the  electrical  activity  of  the  nervous  system.  Oculo-motor  muscles  attached 
around  the  eyeball  allow  our  eyes  to  focus  light  on  the  macula  for  our  sharpest  and  clearest 
vision.  This  high-acuity  area  is  called  foveal  vision,  and  is  distinguished  from  peripheral  vision, 
which  lets  us  see  movement  and  objects  about  180  degrees  fi-om  the  center  of  our  visual  field. 
Peripheral  vision  lets  us  know  we  are  in  a  fog  bank,  for  example,  while  foveal  vision  enables  us 
to  peer  through  a  hole  in  the  fog  looking  for  a  runway. 


Figure  1 .  The  number  of  rods  and  cones  as  a  function  of  distance  from  the  fovea  (Sekular  and  Blake, 

2002,  pg.  70). 

Detection  of  the  motion  of  objects  and  self-motion  are  so  important  that  motion  in  the  visual  field  is 
analyzed  by  a  special  pathway  of  the  visual  system  (Livingstone  &  Hubei,  1988;  Kandel,  1991; 
Ungerleider  &  Haxby,  1994).  According  to  current  views,  multiple  visual  areas  of  the  cerebral 
cortex  receive  a  ventral  stream  of  information  for  object  vision,  and  a  dorsal  stream  for  spatial 
vision  and  motion  (Ungerleider  &  Haxby,  1994).  Using  PET-fMRI  scanners  to  measure  changes  in 
regional  cerebral  blood  flow  in  response  to  visual  stimuli,  studies  with  humans  have  shown  that  the 
ventral  stream  provides  information  about  objects  (the  “what”  is  being  seen),  and  the  dorsal  stream 
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provides  information  on  spatial  vision  or  motion  (“where”  the  object  being  seen  is  located) 
(Ungerleider  &  Haxby,  1994;  Haxby  et  al.,  1994;  Wandell,  1999). 

Sensing  and  controlling  motion  is  now  recognized  as  a  specific  and  complex  process  in  spatial  ori¬ 
entation  behavior.  According  to  recent  studies,  when  people  move  through  tiieir  environment,  the 
visual  world  streams  around  them,  generating  motion  on  the  photoreceptors  of  the  retina,  and  fiiis 
so-called  optic  flow  provides  the  initial  and  compelling  information  for  self-motion  (Duffy  & 

Wurtz,  1995;  Warren  &  Wertheim,  1990).  Although  vision  provides  the  dominant  information  for 
motion  in  humans,  convergent  information  fi'om  vestibular  and  kinesthetic  systems  must  also  be 
integrated  by  the  brain  to  provide  a  more  accurate  and  complete  internal  neural  representation  of 
movement  and  navigation  in  3D  space  (Brandt,  1997;  Lappe,  1997;  Dieterich  et  al.,  1998).  Ac¬ 
cording  to  recent  studies,  this  convergent  multisensory  information  about  motion  may  also  create  a 
spatial  cognitive  map  along  with  head  direction  neurons  and  so-called  path-integration  neurons, 
which  serve  to  detect  body  displacement  and  navigation  in  3D  space  (Berthoz  et  al.,  1995). 

An  important  distinction  in  the  visual  system  is  that  between  focal  and  ambient  vision  (Previc,  1998, 
2000a;  Weinstein  &  Wickais,  1992).  The  distinction  parallels  that  between  the  ventral  and  dorsal 
streams,  and  between  the  foveal  and  peripheral  vision,  but  is  not  identical.  In  particular,  the  foveal- 
peripheral  distinction  is  purely  defined  in  terms  of  retinal  region,  whereas  the  focal-ambient  distinc¬ 
tion  is  fimctionally  defined.  Focal  vision,  which  is  primarily  foveal,  is  used  for  object  detection  and 
recognition,  whereas  ambient  vision,  which  often  includes  large  peripheral  regions,  is  used  for  per¬ 
ceiving  optic  flow  and  orientation,  and  hence  is  critically  involved  in  perception  of  ego-motion 
(Warren  &  Wertheim,  1990;  Previc,  1998).  There  is  some  indication  that  the  two  visual  systems  are 
capable  of  parallel  processing  (Weinstein  &  Wickens,  1992).  In  terms  of  spatial  orientation  in  the 
cockpit,  it  is  assumed  that  a  pilot’s  reference  to  the  tme  horizon,  extending  broadly  across  the  visual 
field,  is  employing  ambient  vision,  whereas  a  distinct  visual  fixation  upon  the  attitude  indicator  is 
probably  more  likely  to  rely  upon  focal  vision  (Previc,  2000a). 

Vestibular  System 

The  parts  of  the  vestibular  system  that  we  will  focus  on  are  the  semicircular  canals,  and  the  oto¬ 
liths  or  vestibular  sacs  in  the  inner  ear  (Figure  2).  The  semicircular  canals  are  filled  with  fluid 
that  indicates  bi-directional  rotation  in  the  pitch,  roll,  and  yaw  axes  via  the  cilia,  or  hair  cells, 
within  each  canal.  The  canals  are  actually  accelerometere  that  sense  changes  in  velocity. 

Unlike  the  seme  of  vision,  the  vestibular  system  does  not  play  a  marked  role  in  conscious  awareness 
of  posture  and  self-movement  (Goldberg  et  al.,  1991;  Kelly,  1991).  This  is  referred  to  as  the  vesti- 
bular  suppression  of  motion  sense  (due  to  the  subservience  of  the  vestibular  system  to  the  visual 
s^tem).  The  vestibular  ^tem  provides  the  neural  mechanism  for  vertical  orientation,  independent 
of  movements  of  the  head  and  body,  and  for  balance,  during  motion.  Normally,  visual-vestibular 
interactions  are  critical  during  motion.  More  specifically,  flie  basic  fimctiom  of  the  vestibulo-ocular 
reflex  (VOR),  a  three-neuron  arc,  are:  to  stabilize  the  visual  world;  to  provide  continuous  visual  and 
vestibular  information  about  the  motion  of  the  eyes,  head,  and  body  for  eye-hand  coordination;  and, 
for  body  posture  (Kelly,  1991).  Figure  3  shows  the  role  of  the  semicircular  canals  in  the  three 
dimensiom  of  fli^t.  The  otoliths,  coimected  to  the  base  of  each  semicircular  canal,  respond  to 
linear  accelerations  and  decelerations. 
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Figure  2.  The  structure  of  the  outer,  middle  and  inner  ears  showing  the  location  of  the  semicircular 

canals  (Rosenzweig  et  al.,  2002,  pg  67). 


Figure  3.  The  role  of  the  semicircular  canals  in  the  perception  of  3-dimensional  motion  (US  Army  Field 

Manual,  2000). 


The  vestibular  system  has  a  long  phylogenetic  history  in  mediating  orientation  to  gravity  in  spatial 
behavior  (Howard,  1986).  However,  imtil  recently,  it  has  appeared  primarily  as  a  silent  partner  in 
regulating  eye  and  head  movements  and  particularly  in  spatial  cognitive  memory  of  self-motion,  or 
body  linear  displacement  in  3D  space  (Berthoz  et  al.,  1995;  Baloh  &  Honrubia,  1990).  Normally, 
vestibular  sensations  do  not  produce  conscious  awareness  of  space  and  gravity  vmless  they  are  in 
conflict  with  information  from  visual  and  proprioceptive  cues. 

At  rest,  the  semicircular  canals  are  shown  as  in  Figure  3  and  no  motion  is  sensed.  As  rotational 
motion  occurs,  the  cilia  (hair  cells)  in  the  canals  bend  in  response  to  the  relative  motion  between  the 
canal  walls,  to  which  the  cilia  are  attached,  and  the  fluid  within  the  canal.  The  fluid’s  mertia 
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provides  initial  resistance  and  braids  the  cilia,  which  informs  the  brain  of  the  acceleration  axis  and 
the  magnitude  of  angular  acceleration.  After  undergoing  sustained  and  constant  angular  motion, 
however  (e.g.,  continuous  yaw  in  a  standard  rate  turn),  the  fluid  within  the  canals  begins  to  rotate  at 
a  velocity  corresponding  to  the  body  and  head.  As  a  consequence,  there  is  no  longer  relative  motion 
between  the  cilia  and  the  fluid  within  the  canals,  the  cilia  are  not  bent,  and  so  no  sensation  of  motion 
is  sent  to  the  brain.  This  phenomenon  is  called  “washout”  and  has  important  implications  for  spatial 
disorientation.  This  washout  occurs  after  approximately  10-15  seconds  of  sustained  rotary  accel¬ 
eration  (Gillingham  &  Previc,  1993). 

Another  important  term  to  SD  research  is  Mulder’s  constant  (DeHart  &  Davis,  2002),  which 
describes  a  threshold  below  whieh  accelerations  are  not  sensed  by  the  human  vestibular  system. 
Mulder’s  constant  is  2°  per  second,  which  makes  it  appear  as  a  velocity,  even  though  it  is  actually 
the  product  of  an  acceleration  and  that  acceleration’s  duration.  The  best  way  to  illustrate  the  mean¬ 
ing  of  Mulder’s  constant  is  with  a  few  examples; 

1 .  If  a  person  experiences  an  acceleration  of  1  for  1  sec,  he  or  she  will  probably  not  sense 
that  acceleration  because  the  product  (17sec)  does  not  exceed  Mulder’s  constant. 

2.  If  the  same  acceleration  occurs  for  3  sec,  however,  it  will  likely  be  detected  (because  the 
product,  3  “/sec,  exceeds  Mulder’s  comtant). 

3.  Even  a  large  acceleration  of  10°/sec^  will  be  felt,  if  its  duration  is  less  than  0.2  sec.  The 
same  exploration  will  be  felt,  if  its  duration  is  0.2  sec  or  greater. 

It  is  important  to  note  that  not  all  humam  have  identical  thresholds,  but  Mulder’s  constant  is  a  good 
generalization  across  a  population  of  people.  Equally  important  is  the  caveat  that  not  all  accelera¬ 
tion  durations  above  Mulder’s  constant  will  be  sensed.  DistreitioiM,  fatigue,  and  other  physiologi¬ 
cal  reasons  may  exist  to  make  a  person  oblivious  to  acceleration  durations  exceeding  Mulder’s 
constant.  It  is  a  good  start,  though,  for  any  SD  modeling  effort,  as  discussed  in  Chapter  IV. 

While  the  semicircular  canals  sense  rotary  motion,  the  otolith  organs  sense  linear  acceleration. 

Here,  in  the  presence  of  linear  acceleration,  the  cilia  within  the  linear  otolith  organs  will  bend  back¬ 
ward,  opposite  to  the  direction  of  acceleration.  This  angle  of  bend,  relative  to  the  orientation  of  the 
organ,  signals  the  degree  of  acceleration.  The  orientation  of  the  cilia  is  as  affected  by  gravitational 
force  as  well  as  by  accelerations  (Howard,  1986).  Hence  a  linear  forward  acceleration  will  produce 
the  same  vestibular  sensation  as  a  tilt  backward,  producing  the  phenomenon  of  the  somatogravic 
illusion  (Gillingham  &  Previc,  1993;  Tokumara  et  al.,  1998),  in  which  pilots  who  execute  a  rapid 
acceleration  (for  example,  on  a  missed  approach)  may  incorrectly  perceive  a  pitch-up  motion. 

The  vestibular  organs  have  been  the  focus  of  many  past  efforts  to  understand  the  SD  that  pilots 
experience  during  fli^t.  The  vestibular  organs  are  important  because  they  constitute  the  key 
sensory  modality  specifically  evolved  to  detect  acceleration  of  the  head  in  inertial  space,  yet  they 
are  not  designed  to  provide  vertical  body  orientation  or  sustained  acceleration  information  within 
the  unusual  sensory-motor  and  force  environments  that  occur  in  flight.  However,  pilots  receive 
training  about  illusions  caused  by  vestibular-visual  conflicts  and  are  taught  to  believe  their  air¬ 
craft  instruments  if  they  experience  such  illusions. 
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Illusions 

Spatial  disorientation  accidents  have  been  referred  to  as  those  in  which  the  pilot  crashes  because 
the  seriousness  of  the  situation  does  not  become  evident  until  it  is  too  late.  There  are  numerous, 
well-documented  illusions  that  are  generated  by  visual-vestibular  conflicts.  This  section  will 
describe  some  of  the  kinds  of  illusions  that  result  from  the  unique  environments  produced  by 
human  flight. 

Leans 

The  most  common  SD  illusion  in  flight  is  the  Leans  (Holmes  et  al.,  2003;  Benson,  1988),  which 
entails  an  erroneous  feeling  of  roll.  A  typical  case  occurs  in  the  following  scenario  (Gillingham 
&  Previc,  1993):  In  IMC,  the  pilot  has  very  slowly  entered  a  turn,  perhaps  unknowingly,  at  a 
sub-threshold  rate  of  less  than  2°/sec  (i.e.,  below  Mulder’s  constant)  so  that  the  semicircular 
canals  provide  no  sense  of  rotation.  If  the  pilot  then  becomes  consciously  aware  of  the  aircraft’s 
bank  angle  (e.g.,  by  looking  at  the  instruments),  and  intentionally  returns  the  bank  to  the  true 
level  attitude,  he  or  she  will  now  receive  a  vestibular  sensation  of  an  opposite  bank.  If  the  pilot 
continues  to  rely  upon  flight  instruments  to  maintain  a  level  attitude,  he  or  she  may  also  lean  in 
the  orientation  of  the  incorrectly  perceived  upright  (hence  the  illusion’s  name).  If  the  pilot  does 
not  rely  upon  the  instruments  at  this  point,  but  rather  the  intuitive  (vestibular)  signal  of  upright, 
he  or  she  will  return  the  aircraft  to  its  original  bank  angle.  Without  awareness  and  conscious  cor¬ 
rection,  the  bank  will  lead  to  a  gradual  pitch  down  attitude,  a  loss  of  altitude,  and  an  increase  in 
airspeed.  This  side-to-side  seesawing  process  of  correction  and  re-entry  may  continue  until  the 
pilot  is  so  disoriented  that  recovery  to  straight-and-level  flight  is  difficult,  if  not  impossible  (a 
Type  3  SD  event).  This  may  lead  to  a  progression  into  a  more  severe  loss  of  altitude,  which  ends 
with  ground  impact. 

Coriolis 

The  pilot ’s  inability  to  control  the  airplane  because  of  spatial  disorientation,  exacerbated  by 
failure  of  the  primary  attitude  indicator,  night  IMC,  adverse  weather  including  turbulence  and 
noise  from  rain,  were  the  cause  and  contributing  factors  of  the  Oct.  2000  crash  of  a  Cessna  335 
that  killed  Missouri  Governor  Mel  Carnahan  and  all  others  aboard  the  aircraft  (NTSB,  2002). 

The  fatal  accident  that  killed  Governor  Carnahan  and  his  entourage  is  a  classic  and  tragic  exam¬ 
ple  of  an  SD  scenario.  We  do  not  know  whether  it  was  unrecognized  (Type  1)  or  recognized 
(Type  2),  only  that  the  pilot  seemed  to  pitch  down  and  to  the  right  during  his  attempt  to  return 
and  land  after  an  aircraft  instrument  malfunction  shortly  after  takeoff  The  NTSB  found  that  the 
primary  attitude  indicator  (AI)  failed  in  instrument  meteorological  conditions  (IMC).  However, 
the  co-pilot’s  attitude  indicator  was  working  at  the  time  of  the  crash.  The  NTSB  report  indicates 
that  the  pilot  would  logically  try  to  use  the  co-pilot’s  AI,  located  to  the  pilot’s  right.  Using  that 
AI  and  then  checking  his  own  instruments  for  heading  and  altitude,  the  pilot’s  head  motions, 
coupled  with  the  airplane’s  turn  back  toward  the  takeoff  runway  would  induce  a  Coriolis 
illusion. 

The  FAA’s  Airman’s  Information  Manual  (ASA,  1995)  describes  the  Coriolis  illusion  as  the 
“most  overwhelming  of  all  illusions  in  flight”  (page  645).  It  occurs  during  “a  prolonged 
constant-rate  turn  that  has  ceased  stimulating  the. . .  [vestibular]  system,”  (i.e.,  washout)  when 
there  is  an  abrupt  head  movement  that  creates  “the  illusion  of  rotation  or  movement  in  an  entirely 
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different  axis,”  The  resulting  tumbling  sensation  is  nearly  incapacitating.  The  Coriolis  illusion 
often  results  in  the  pilot’s  loss  of  aircraft  control,  which  can  have  catastrophic  results  if  there  is 
no  oftier  pilot  to  assume  control.  The  power  of  the  Coriolis  illusion  to  deceive  is  further  sup¬ 
ported  by  the  finding  in  a  recent  survey  that  it  was  the  most  prevalent  of  those  found  in  141 
pilots  attending  a  course  at  Randolph  AFB  in  1997-98  (Sipes  &  Lessard,  1999). 

The  Coriolis  illusion  was  selected  as  one  of  the  prototypical  illusions  in  this  project  because  it  is 
among  the  most  common  flight  illusions  (#5  in  the  list  compiled  by  Holmes  et  al.,  2003)  and  is 
often  fatal.  It  causes  overwhelming  disorientation  and  so  is  an  appropriate  illusion  to  combat. 

Conclusion 

In  summary,  the  physiological  circumstances  underlying  many  spatial  disorientation  illusions  are 
as  follows:  Aircraft  motion,  sometimes  coupled  with  head  motion,  provides  illusory  vestibular 
signals  to  the  brain.  Normally,  when  salient  visual  cues  are  available,  visual  dominance  will 
allow  the  vestibular  cues  to  signal  the  correct  orientation  and  motion.  However,  when  the  visual 
cues  are  degraded  or  missing,  the  compelling  vestibular  signals  will  dominate,  and  illusions  will 
occur.  Knowledge  of  how  these  illusions  begin  and  progress  provides  suggestions  for  counter¬ 
measures,  which  we  address  in  the  following  chapter. 
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III.  Countermeasures:  Displays  &  Other  Technologies 

This  chapter  describes  a  multisensory  approach  to  preventing,  minimizing,  or  compensating  for 
pilot  spatial  disorientation.  Since  human  spatial  orientation  has  such  a  large  visual  component, 
we  describe  visual  displays  in  some  depth.  Audio  and  tactile  displays  follow,  and  then  an  intelli¬ 
gent  system  approach  to  applying  the  various  SD  countermeasures.  All  of  these  technologies  fit 
within  a  layered  approach  to  combating  SD. 

Background 

We  suggest  a  layered  approach  to  improving  pilot  attitude  awareness  and  reducing  the  negative 
consequences  of  SD  events.  Improving  current  cockpit  displays  (i.e.,  head-down,  head-up,  and 
helmet-mounted)  and  incorporating  new  technologies  (e.g.,  3D  audio,  tactile)  requires  a  method¬ 
ology  for  integrating  and  testing  all  of  the  salient  technologies  and  techniques  to  determine 
which  ones  are  the  best  at  enhancing  attitude  awareness,  preventing  SD  events,  and  minimizing 
the  impact  of  any  SD  events  that  do  occur.  A  layered  approach  begins  with  pilot  training  and 
mission  pre-briefings,  and  extends  through  improved  display  technologies  and  recovery  aiding, 
into  post-flight  data  analyses  to  better  understand,  categorize,  and  track  SD  events  to  determine 
the  actual  operational  effectiveness  of  the  various  techniques  advocated  to  improve  SD  statistics. 

Recent  research  has  explored  new  approaches  to  addressing  SD:  better  attitude  information  for 
head-down,  head-up,  and  helmet-mounted  displays;  3D  audio  displays;  and,  tactile  displays. 
Which  of  these  approaches,  or  combination  of  approaches,  is  best  at  reducing  SD?  Because  the 
potential  of  each  individual  technology  to  reduce  SD  has  not  been  established  (theoretically  or 
empirically),  it  is  difficult  to  postulate  the  “best”  combination.  Rather,  it  seems  that  whichever 
displays  are  best  in  a  given  set  of  circumstances  at  helping  pilots  to  recognize  and  recover  fi-om 
that  SD  event,  then  those  are  the  displays  that  should  be  used. 

How  do  researchers  know  which  displays  are  best  for  different  SD  scenarios?  There  is  nothing 
in  the  current  literature  to  suggest  an  answer;  therefore,  it  will  be  difficult  to  devise  the  best 
combination.  In  fact,  a  best  combination  or  solution  may  not  be  possible  for  the  foreseeable 
future,  due  to  individual  pilot  and  mission  susceptibility,  risk  factors,  and  circumstances.  But, 
we  can  theoretically  and  empirically  determine  good  combinations  of  inputs  to  help  combat  SD. 

Determining  the  best  approach  for  converting  Type  1  into  Type  2  SD  events  should  take  a  human 
cognition  and  workload  modeling  approach.  When  visual  senses  are  overloaded  or  deemed 
unreliable,  the  audio  channel  may  be  useful.^  A  tactile  (vest)  attitude  display  may  also  help,  but 
research  into  tactile  displays  is  in  its  infancy.^  A  presently  untested  sense  for  SD  is  the  sense  of 
smell.  Smells  are  very  compelling  and  may  be  useful  for  Type  3  (incapacitating)  SD  to  help 
pilots  “snap  out  of  it.”  Such  an  olfactory  system  will  need  sensors  to  know  that  the  pilot  is  inca¬ 
pacitated,  and  the  smell  used  must  be  penetrating,  harmless,  and  easily  carried  aboard  aircraft. 


*  It  is  common  aviation  knowledge  that  air  traffic  controllers  have  talked  disoriented  pilots  through  recoveries.  Tlie 
proposed  PI  has  an  audiotape  of  a  controller  telling  a  pilot  to  “release  the  controls”  to  recover  from  a  graveyard 
spiral. 

®  Vibrotactile  transducers:  Utilizing  tactile  displays  to  improve  performance  in  spatial  tasks  in  aerospace,  land  and 
sea  environments.  ht^://www.navysbir.brtrc.com/cap/briefingsadmin/ea.asp. 
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When  the  pilot  does  not  recognize  an  SD  event  (Type  1),  intelligent  aiding  systems  may  help. 

For  example,  audio  recovery  instructions  have  shown  some  early  utility  with  SD  events  in  simu¬ 
lators.^  They  key  is  for  the  aiding  system  to  recognize  when  SD  is  causing  an  unusual  attitude  or 
dangerous  situation.  The  emphasis  of  such  systems  ought  to  be  on  preventing  the  bad  conse¬ 
quences  of  SD  events,  since  it  will  be  virtually  impossible  to  prevent  all  SD  events  due  to  the 
normal  physiological  response  to  most  events  that  induce  SD  (as  described  in  the  previous 
chapter). 

Consequently,  the  best  approach  is  a  layered  one  that  seeks  to  enhance  attitude  awareness  and 
combat  SD  via  multiple  techniques:  training  improvements,  operational  awareness,  better  dis¬ 
plays  (visual,  auditory,  tactile,  olfactory),  intelligent  automation,  and  rigorous  post-flight  SD 
event  analyses  to  better  understand  and  characterize  SD  events,  which  should  also  indicate  the 
operational  effectiveness  of  the  various  SD-combating  techniques. 

Training  Improvements 

Training  can  help  pilots  recognize  the  types  of  circumstances  that  lead  to  SD  so  that  they  will  be 
vigilant  for  the  same  or  similar  circumstances  (Gillingham  &  Previc,  1993).  Training  also  needs 
to  be  enhanced  and  should  include  classroom  iscussion,  simulator  and  centrifuge  training,^  and 
aircraft  demonstrations.  While  there  is  some  risk  to  aircraft  demonstrations,  the  risk  of  not  doing 
so  is  reflected  in  the  rates  of  SD  accidents  in  military  and  civil  aviation  over  many  decades. 
Clearly,  not  all  SD  event  categories  could  be  demonstrated,  but  a  significant  cross-section  could 
be.  Teaching  SD  situation  recognition  is  a  major  contribution  to  prevention.’  And,  pilots  who 
are  more  vigilant  for  SD-inducing  circumstances  are  better  equipped  to  avoid  or  successfully 
combat  them.  It  is  also  important  to  teach  recovery  techniques  for  each  SD  situation  to  reduce 
the  occurrences  of  Type  3  SD  events.  The  most  prevalent  factor  in  USAF  SD  incidents  is  atten¬ 
tion  management  (Davenport,  2000),  a  skill  that  can  (and  should)  be  improved  with  training 
(Gopher,  Weil,  &  Barakeit,  1994). 

Operational  Awareness 

Pre-briefings  could  reduce  some  SD  event  severity.  Certainly,  pre-briefings  could  increase  the 
odds  of  any  SD  event  being  Type  2  rather  than  Type  1.  Since  the  environmental  conditions  that 
increase  susceptibility  to  SD  are  known,  mission  pre-briefings  will  be  timely  reminders  to  pilots, 
thus  reducing  the  likelihood  of  an  SD  event  being  Type  2  or  3.  The  Indian  Air  Force  has  devel¬ 
oped  a  checklist  enumerating  SD  threats,  and  has  even  anticipated  the  increased  threat  of  SD 
fi-om  supermaneuvers,  in  such  aircraft  as  the  F-22,  F-35,  V-22,  and  Apache  helicopter  (Gomez, 
2002). 

US  Air  Force  statistics  about  contributing  factors  to  SD  incidents  point  to  conditions  in  which 
the  pilot  is  highly  susceptible  to  SD.  All  of  this  knowledge  and  experience  should  be  highlighted 
during  mission  planning  and  pre-briefings  so  that  pilots  can  be  vigilant.  Weather  conditions, 
moonless  night  flints  over  the  desert,  and  night  vision  goggle  (NV G)  use,  are  all  known  to  be 


’  Personal  communication  with  Dr.  Kristen  Liggett  (11/21  /02) :  “. . .  ‘talking’  the  pilot  through  a  recovery  is 
gomstliitig  we  jiKt  recently  tried  in  flie  lab....  The  date  is  still  being  analyzed,  but... they  [the  pilot  subjects]  followed 
the  instructions  and  had  successful  recoveries.” 

^  There  is  a  new  type  of  simulator  for  such  training;  see  Aviation  Week  &  Space  Technology,  12/2/02  issue,  pg.  60. 

®  Preventing  disorientation  mishaps,  http://www.spatiald.wpafb.af.mil/prevention.asp. 
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risk  factors  (Berkley  &  Martin,  2000).  Reminding  pilots  of  the  relevant  mission  risk  factors,  and 
what  to  do  if  SD  occurs,  refreshes  their  earlier  training  -  another  important  layer  in  a  compre¬ 
hensive  approach  to  mitigating  the  effects  of  SD. 

Display  Improvements 

Because  the  primary  source  of  information  to  the  pilot  is  visual,  because  the  visual  modality  is 
the  most  compatible  modality  for  presenting  spatial  information  relevant  to  attitude  (Wickens  et 
ah,  1983),  and  because  SD  incidents  are  most  prevalent  in  IMC  or  at  night  (when  the  view  of  the 
true  horizon  is  obscured  or  gone),  it  is  not  surprising  that  a  key  countermeasure  in  combating  SD 
is  in  the  design  of  effective  visual  displays.  Sadly,  many  advances  in  this  direction  are  lacking, 
because  design  has  generally  focused  on  providing  the  necessary  information  to  support  spatial 
orientation,  but  not  on  the  ideal  format  in  which  that  information  should  be  depicted  most  natu¬ 
rally  or  intuitively.  One  of  the  major  culprits  appears  to  be  the  focus  on  using  the  round  dial  or 
“steam  gauge”  display,  even  as  cockpit  displays  have  evolved  to  include  computer  generated 
imagery,  where  the  electromechanical  constraints  associated  with  the  round  dial  are  no  longer 
relevant.  In  the  pages  below,  we  discuss  some  of  the  display  formatting  issues  and  ideas  that  are 
relevant  for  effective  support  of  spatial  orientation,  in  conventional  head-down  instruments,  and 
in  head-up  and  helmet-mounted  displays. 

Frame  of  reference  for  the  attitude  indicator 

Clearly,  in  IMC,  a  pilot’s  perception  of  the  upright  must  be  supported  by  a  well-designed  attitude 
indicator  (AI),  which  can  support  the  intuitive  grasp  of  the  orientation  of  the  horizon.  Much  of 
the  research  on  AI  design  has  focused  on  the  question  of  whether  the  aircraft  symbol  or  the  hori¬ 
zon  line  should  move  as  a  function  of  aircraft  bank  and  pitch.  The  conventional  design,  pre¬ 
senting  a  moving  horizon,  is  argued  to  be  congruent  with  the  pilot’s  mental  model  or  mental  rep¬ 
resentation  of  the  forward  horizon  appearing  to  slant  in  one  direction,  which  is  opposite  to  the 
direction  of  the  aircraft’s  actual  bank.  Thus,  this  AI  design  corresponds  to  what  Roscoe  (1968) 
describes  as  the  principle  of  pictorial  realism  (see  also  Wickens  &  Hollands,  2000;  Wickens, 
2003). 

In  contrast,  the  principle  of  the  moving  part,  also  associated  with  Roscoe’s  (1968)  work,  pro¬ 
poses  that  the  moving  element  on  a  display  should  correspond  to  the  element  that  moves  in  the 
pilot’s  mental  model  of  the  aircraft,  and  should  move  in  the  same  direction  as  that  mental  repre¬ 
sentation.  Because  the  pilot’s  mental  model  represents  the  aircraft  moving  in  a  world  that  is 
stable  (Johnson  &  Roscoe,  1972;  Kovalenko,  1991;  Merryman  &  Cacioppo,  1997;  Previc  & 
Ercoline,  1999;  Previc,  2000a),  this  principle  suggests  that  the  aircraft  should  be  depicted  as  the 
moving  element  on  a  display,  and,  for  example,  when  the  aircraft  climbs  or  pitches  up,  this  ele¬ 
ment  should  move  upward.  The  moving  horizon  of  the  attitude  indicator  in  Figure  4a  violates 
this  principle,  since  the  moving  element  (horizon)  moves  upward  to  signal  that  the  aircraft  is 
pitching  downward,  and  rotates  to  the  right  (or  left)  to  signal  that  the  aircraft  is  rotating  left  (or 
right).  To  conform  to  the  principle,  the  airplane  should  move  around  a  stable  horizon,  as  in 
Figure  4b. 
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displays  show  a  left  bank.  Low  frequency  return  to  steady  state  is  indicated  by  arrows  in  (c)  (Wickens  & 

Hollands,  2000,  pg.  137). 

However  the  conventional  design  to  configure  the  AI  as  a  moving  horizon,  rather  than  as  a  mov¬ 
ing  aircraft,  is  one  that  can  be  argued  to  better  satisfy  the  principle  of  pictorial  realism,  since, 
when  the  pilot  looks  outside,  the  moving  horizon  display  better  corresponds  with  what  is  seen 
looking  forward  fi-om  the  cockpit.  Roscoe  (2002)  has  argued  that  the  consequences  of  such  vio¬ 
lations  of  the  principle  of  the  moving  part,  are  possible  confusions  leading  to  spatial  disorienta¬ 
tion,  when,  for  example,  a  downward  display  movement  is  incorrectly  interpreted  as  a  downward 
aircraft  movement;  or,  when  a  clockwise  rotation  of  the  horizon  on  the  AI,  is  misinterpreted  as  a 
clockwise  rotation  (right  bank)  of  the  aircraft  (Roscoe,  1968;  Kovalenko,  1991),  If  these  misin¬ 
terpretations  are  followed  immediately  by  the  correct-appearing  (but  actually  backward)  control 
corrections,  then  the  off-attitude  state  of  the  aircraft  will  be  amplified,  rather  than  nullified,  exac¬ 
erbating  the  situation. 

The  results  of  earlier  studies  by  Roscoe  and  his  colleagues  (Beringer,  Williges,  &  Roscoe,  1975; 
Roscoe  &  Williges,  1975),  and  Kovalenko  (1991)  comparing  moving  aircraft  with  moving  hori¬ 
zon  attitude  indicators  h^  led  to  considerable  debate  about  the  ideal  configuration  of  this  im¬ 
portant  instrument  (Previc  &  Ercoline,  1999),  to  conform  to  the  one  principle  (moving  part)  or 
the  other  (pictorial  realism).  A  review  of  the  literature  suggests  that  the  moving  aircraft  display 
is  more  effective  than  the  moving  horizon  display  for  novice  pilots  with  no  prior  experience  on 
either,  and  is  no  less  effective,  even  for  experienced  pilots  who  have  had  prior  flight  experience 
with  the  moving  horizon  display  (Previc  &  Ercoline,  1999;  Cohen  et  al.,  2001).  The  advantage 
of  the  moving  aircraft  will  be  amplified  in  IMC  (where  there  is  no  true  horizon  visible),  and 
diminished  in  VMC  ferryman  &  Cacioppo,  1997;  Gallimore  et  al.,  2000). 

The  overall  benefit  for  a  stable  horizon  may  result,  in  part,  firom  the  pilot’s  reflexive  tendency  to 
orient  the  head  to  the  outside  horizon  as  the  airplane  banks  (i.e.,  to  visually  stabilize  the  horizon), 
a  phenomenon  called  the  opto-kinetic  cervical  reflex  (OKCR)  (Merryman  &  Cacioppo,  1997; 
Previc  &  Ercoline,  1999;  Previc,  2000a;  Smith  et  al.,  1997).  If  the  head  rotates  counter  to  tiie 
airplane  when  the  latter  banks,  so  that  the  line  connecting  the  two  eyes  parallels  the  horizon,  then 
the  eyes  will  actually  see  a  stable  horizon  as  the  airplane  banks,  and  those  aspects  of  the  airplane 
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that  are  in  the  visual  field  will  he  seen  to  rotate;  that  is,  a  direct  perception  will  be  one  of  a 
moving  airplane,  corresponding  to  what  is  seen  on  a  moving  airplane  AI.  While  the  OKCR  is  a 
phenomenon  that  is  primarily  manifest  in  VMC,  when  the  true  horizon  is  visible  (Gallimore  et 
ah,  2000),  the  underlying  tendency  to  orient  the  head  to  the  perceived  upright,  seen  so  strongly 
when  the  horizon  is  visible,  is  also  manifest  when  the  horizon  is  inferred  from  vestibular 
feedback  in  IMC,  as  manifest  in  the  Leans,  discussed  in  Chapter  11. 

The  role  of  OKCR  in  supporting  the  moving  aircraft  bank  perception,  might  suggest  that  this 
world-referenced  frame  (moving  aircraft)  is  more  pronoimced  in  bank  than  in  pitch,  which  is  the 
case  (Merryman  &  Cacioppo,  1997;  Patterson  et  ah,  1997;  Smith  et  ah,  1997). 

While  the  debate  is  not  likely  to  lead  to  the  redesign  of  the  head-down  attitude  indicator  with  a 
moving  aircraft,  it  is  possible  that  that  certain  design  solutions  can  capture  elements  of  both  prin¬ 
ciples.  The  concept  of  a  frequency-separated  display  (Figure  4c)  is  one  such  solution,  in  which 
rapid  (i.e.,  high  frequency)  rotation  of  the  aircraft,  as  when  entering  a  turn,  leads  to  a  rotation  of 
the  aircraft  symbol  on  the  AI  in  the  same  direction  as  the  true  rotation  of  the  aircraft.  Then,  after 
the  aircraft  stabilizes  at  the  new  bank  angle  and  holds  this  steady  state  for  some  time  (i.e.,  low 
frequency),  the  AI  horizon  line  gradually  rotates  to  correspond  to  the  true  horizon,  and  the  air¬ 
craft  symbol,  in  turn,  rotates  back  to  an  upright  position.  The  reverse  control,  implemented  to 
bring  the  aircraft  back  to  straight  and  level,  will  reverse  this  sequence  of  motion. 

Another  solution  is  the  3D  pathway  display,  or  highway  in  the  sky  (HITS)  display.  This  display 
provides  a  prediction  of  the  future  location  of  the  aircraft,  which  conforms  to  the  principle  of  the 
moving  part,  even  as  the  attitude  indicator  itself  conforms  to  the  principle  of  pictorial  realism 
(Fadden,  Ververs,  «&  Wickens,  2001;  Alexander,  Wickens,  &  Hardy,  2003).  That  is,  a  right  bank 
of  the  aircraft  will  initiate  an  immediate  rightward  movement  of  the  predictor  symbol  on  the  dis¬ 
play,  creating  an  immediate  and  intuitive  perception  of  a  right  turn  (i.e.,  bank),  even  if  the  artifi¬ 
cial  horizon  rotates  to  the  left.  Roscoe  (2002)  has  argued  that  such  a  design  may  mitigate  the 
concerns  for  roll  reversals. 

Finally,  other  research  on  horizon  representation  has  focused  on  a  moving  horizon  format,  but 
distributed  across  a  wide  range  of  space  in  the  cockpit,  so  that  it  can  be  perceived  not  just  with 
focal  and  foveal  vision  on  a  small  attitude  indicator,  but  also  with  ambient  vision  across  a  wide 
range  of  the  visual  field,  making  it  more  resistant  to  attentional  tunneling,  just  as  we  view  the 
normal  horizon  for  orientation  information  (Previc,  2000a).  One  means  of  doing  so  is  th-rough 
laser  projection  of  the  so-called  Malcomb  Horizon,  projected  in  space  from  behind  the  pilot 
(Gawron  &  Knotts,  1984).  Current  technological  limitations  have  prevented  these  from  realizing 
success  in  flight  tests  (see  Stokes,  Wickens,  &  Kite,  1990  for  a  review).  A  second  way  of 
accomplishing  this  has  been  proposed  as  a  Background  Attitude  Indicator  (Liggett,  Reising,  & 
Hartsock,  1999),  which  presents  the  horizon  spanning  across  smaller  panel  displays.  Both  of 
these  techniques  allow  perception  of  the  horizon  as  the  head  is  rotated.  A  third  approach  is  to 
present  the  horizon  on  a  helmet-mounted  display  (HMD).  We  consider  this  issue  below  in  the 
broader  context  of  presenting  overlaid  information  on  either  an  HMD  or  a  head-up  display 
(HUD). 
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HUDs  &  helmet-mounted  displays 

The  issue  of  the  frame  of  reference  for  the  attitude  indicator  on  a  see-through  display  (HUD  or 
HMD)  adds  additional  complications.  For  the  HUD,  the  problems  of  motion  incompatibility 
between  a  moving  aircraft  display,  and  the  moving  horizon  seen  outside  will  be  amplified  in 
VMC,  relative  to  a  head-down  presentation,  because  the  two  different  motion  frames  will  be 
viewed  directly  superimposed  on  each  other.  For  the  HMD,  which  we  discuss  in  some  detail,  the 
problem  of  frame  of  reference  for  attitude  representation  and  spatial  disorientation  become  even 
more  complex,  when  the  HMD  is  used  for  off-boresight  viewing. 

The  design  of  the  HMD  (Melzer  &  Moffitt,  1997;  Wickens  &  Rose,  2001)  must  consider  two 
general  human  factors  issues,  which  we  shall  address  below: 

(1)  What  is  the  advantage  (or  cost)  of  presenting  information  on  an  HMD,  rather  than  on  a 
conventional  fixed  display? 

(2)  How  should  the  imagery  on  the  HMD  be  presented? 

It  turns  out  that  the  answer  to  the  second  question  greatly  depends  on  both  the  task  and  the  frame 
of  reference  (FOR)  with  which  the  imagery  is  presented,  and  so  we  shall  focus  on  three  different 
frames  of  reference  for  presenting  HMD  information.  It  further  turns  out  that  the  choice  of  an 
HMD  to  present  information  (and  the  choice  of  a  particular  FOR),  also  generates  certain 
spawned  costs  analogous  to  ftiose  encountered  when  choosing  a  3D  (HITS)  display. 

The  HMD  has  two  distinct  benefits  relative  to  the  HUD: 

(1)  Allowing  continuous  viewing  of  instrumentation  across  a  wide  range  of  head  movements 
(in  particular^  when  engaged  in  off-axis  viewing).  This  potentially  allows  instant  access 
to  attitude  information,  and  can  therefore  potentially  mitigate  concerns  of  attitude 
awareness. 

(2)  Enabling  a  much  wider  range  of  visual  space  within  which  conformal  imagery  can  be 
displayed. 

These  two  benefits  are  generally  associated  with  different  frames  of  reference  of  HMD  imagery 
(Wickens  et  al.,  2001).  Specific  images  (usually  non-conformal)  which  are  viewable  independ¬ 
ently  of  the  orientation  of  the  head,  are  said  to  be  screen-referenced,  since  the  coordinates  upon 
which  those  images  are  drawn,  are  defined  exclusively  by  X-Y  coordinates  on  the  HMD  surface. 
In  control,  conformal  imagery  on  an  HMD  is  said  to  be  world-referenced,  since  it  overlays  dis¬ 
tinct  positions  in  the  world  and  its  location  on  the  screen  will  be  contingent  upon  the  momentary 
orientation  or  movement  of  the  head.  World  referencing  on  a  see-through  HMD  creates  what  is 
known  as  augmented  reality  (Milgram  &  Colquhoun,  1999).  World  referencing  on  an  opaque 
HMD,  while  not  discussed  in  detail  here,  can  be  used  to  create  the  virtual  cockpit  (Barfield  & 
Furness,  1995),  which  mi^t  be  used  for  training  purposes,  or  for  flying  remotely  piloted  vehi¬ 
cles. 

A  schematic  rqjresentation  of  the  view,  seen  by  pilots  in  a  screen-referenced  and  a  world-refer¬ 
enced  HMD  is  shown  Figures  5a  and  5c,  for  pilots  who  are  looking  to  the  right  and  see  the  indi¬ 
cator  of  their  aircraft  in  a  right  bank.  In  addition  to  the  world-  and  screen-referenced  frames 
shown  in  Figure  5,  HMDs  allow  depiction  of  a  ftiird  frame  of  reference  -  the  aircraft,  or  aircraft- 
referenced  symbology.  Here  the  symbology  is  drawn  at  a  fixed  location  relative  to  the  axis  of 
the  aircraft.  An  example  would  be  a  set  of  flight  instruments  that  are  depicted  to  appear  at  the 
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normal  location  of  the  HUD.  This  view  is  shown  in  panel  Figure  5b,  representing  the  turning 
head. 

While  the  distinction  between  these  three  frames  of  reference  for  HMDs  is  every  bit  as  critical  as 
is  the  distinction  between  conformal  and  non-conformal  imagery  for  HUDs,  there  are  in  fact  very 
few  studies  that  have  contrasted  them,  to  allow  conclusions  of  “which  is  better”  to  be  drawn. 
Taylor  and  Kuchar  (2000)  compared  screen-referenced  to  aircraft-referenced  symbology  for 
attitude  recovery  for  fighter  pilots.  Their  experiments  required  pilots  to  look  off  axis  with  the 
HMD  while  at  an  unusual  attitude,  and,  upon  hearing  an  auditory  cue,  immediately  recover  to 
wings-level  flight.  The  investigators  found  that  the  airplane-referenced  display  imposed  a  delay 
in  recovery,  as  the  pilots  needed  to  return  the  head  to  a  forward  viewing  position  before  initiating 
the  maneuver,  whereas  with  the  screen-referenced  display,  pilots  could  (and  did)  initiate  their 
recovery  before  rotating  their  head  forward. 

Herdman  and  colleagues  (2000)  also  compared  a  screen-referenced  HMD  with  an  aircraft- 
referenced  HMD  and  contrasted  these  with  a  mixed  display  containing  both.  Their  rotorcraft 
pilots  engaged  in  a  forward  flying  maneuver.  In  the  mixed  display,  attitude  information  re¬ 
mained  aircraft-referenced,  so  that  it  would  be  conformal  with  the  true  horizon  when  the  pilot 
looked  forward,  but  drop  out  of  sight  (like  a  conformal  HUD)  when  the  pilot  looked  off  axis. 

The  investigators  found  a  general  advantage  for  screen-referenced  information.  In  the  mixed 
condition,  the  presence  of  both  kinds  of  symbology  on  a  single  display,  moving  differently  as  the 
pilot’s  head  rotated,  did  not  appear  to  disrupt  performance,  because  of  the  ability  of  pilots  to 
cognitively  separate  the  two  domains  on  the  basis  of  motion  cues. 

In  both  of  the  above  experiments,  aircraft-referenced  attitude  information  was  also  configured  to 
be  partially  conformal  with  the  outside  world,  in  the  sense  that,  when  the  pilot  looks  forward  and 
varies  attitude,  the  behavior  of  the  instrument  horizon  would  mimic  that  of  the  true  horizon,  as 
would  be  the  case  of  a  partially  conformal  HUD.  This  raises  the  question  as  to  whether  such 
conformal  but  screen-referenced  displays  (like  the  AI  shown  in  Figure  5  a)  would  be  disruptive  to 
aircraft  control  if  they  are  viewed  off  axis;  that  is,  a  contrast  between  panels  (a)  and  (c)  in  Figure 
5,  as  these  would  be  used  for  attitude  control.  Both  panels  depict  a  plane  banked  to  the  right; 
however,  the  conformal  world-referenced  horizon  symbol  shown  in  Figure  5c  depicts  the  horizon 
as  it  would  look  to  a  pilot  looking  outward  along  the  right  wing  of  a  right-banked  aircraft. 

Again,  there  is  little  data  to  compare  the  efficacy  of  these  two  frames  of  reference. 

Haworth  and  Seery  (1992)  compared  the  two  frames,  across  a  range  of  typical  tasks  for  the  rotor- 
craft  pilot  in  a  full  mission  simulation,  while  pilots  looked  off-axis,  as  they  were  engaged  in 
search  for  targets  and  maintaining  awareness  of  terrain  features.  They  found  that  neither  display 
was  terribly  effective  for  unusual  attitude  recovery  during  off-axis  viewing  with  both  showing  a 
large  number  of  initial  control  actions  in  the  wrong  direction;  the  world-referenced  display 
showed  more  errors.  In  the  world-referenced  frame  (Figure  5c),  it  is  easy  to  see  how  vertical 
movement  of  the  horizon  line,  signaling  a  bank  change,  could  be  misinterpreted  as  a  pitch 
change.  However,  Haworth  and  Seery  (1992)  also  foxmd  that  for  tasks  involving  perception  of 
the  simulated  outside  world  (e.g.,  altitude  and  terrain  estimation,  contour  flight),  the  world-refer¬ 
enced  symbology  either  supported  better  performance  or  was  more  favorably  rated  than  was  the 
screen-referenced  symbology.  This  makes  sense,  given  the  total  conformality  (scene  linking)  of 
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the  world-referenced  attitude  with  the  outside  world  view,  a  particular  advantage  in  low-level 
flight. 


Figure  5,  Four  views  depicting  three  different  frames  of  reference  on  a  helmet-mounted  display.  The 
arrow  at  the  top  shovre  the  direction  of  flight  and  axis  of  the  aircraft.  The  figures  in  the  middle  row  show 
the  orientation  of  the  head  and  the  HMD.  The  figures  on  the  bottom  show  the  view  on  the  HMD  screen  of 
an  aircraft  banked  to  the  right,  (a)  Screen-referenced,  (b)  Aircraft-referenced  (two  views  as  the  head 
turns),  (c)  Worid-referenced.  The  gray  band  represents  the  ground  and  the  white  band  represents  the 
air.  The  dividing  horizon  line  would  move  downward  if  the  pilot  returned  to  a  wings-level  attitude. 

The  two  slightly  contradictory  trends  (one  favoring  e^h  finme)  observed  by  Haworth  and  Seery 
(1992)  are  consistent  with  two  emerging  findings.  First,  as  we  described  earlier,  the  world  refer¬ 
encing  of  the  conformal  symbology  set  may  facilitate  the  division  of  attention  between  the  in- 
stniments  and  the  outside  world,  eliminating  any  visual  conflict  between  the  two  visual  sources, 
and,  like  scene  linking,  serve  to  better  fuse  the  two  views  (Levy  et  al.,  1998;  Fadden  et  al.,  2001). 
This  fusion  would  not  occur  as  a  result  of  ttie  conflicting  motion  patterns  when  the  screen-refer¬ 
enced  symbology  is  viewed  off-axis.  Second,  there  is  some  evidence  that  the  interpretation  of 
displayed  motion  when  viewed  off-axis,  is  easily  and  automatically  mentally  transformed  to  a 
forward  viewing  axis,  just  as  the  head  itself  naturally  rotates  to  the  canonical  forward  view 
relative  to  the  trunk  (Wickens  et  al.,  2003;  Worringham  &  Beringer,  1989;  Previc,  1998). 

Hence,  a  display  that  presents  motion  that  would  be  un-transformed  when  viewed  in  a  forward 
orientation  should  be  more  rapidly  and  correctly  interpreted,  consistent  with  the  findings  of 
Haworth  and  Seery  (1992)  favoring  the  screen-referenced  symbology  for  attitude  recovery. 

A  study  by  Cohen  et  al.  (2001)  is  also  consistent  with  the  latter  conclusion.  Although  the  inves¬ 
tigators  did  not  employ  a  true  HMD,  they  positioned  a  standard  screen-referenced  attitude  indi¬ 
cator  in  either  a  forward  view  or  in  a  90-degree  off-axis  viewing  plane,  relative  to  the  orientation 
of  the  pilot’s  trunk  (and  control  axis).  For  both  skilled  pilots  and  naive  non-pilots  Cohen  et  al. 
(2001)  observed  no  cost  to  ofif-axis  viewing  in  a  standard  (pitch  and  roll)  tracking  task.  That  is, 
even  when  the  instrumentation  was  placed  off  axis,  pilots  were  able  to  mentally  rotate  the  fi-ame 
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of  reference  to  a  forward  view  automatically  with  no  cost.  This  equality  across  viewing  loca¬ 
tions  was  observed  independently  of  whether  an  inside  out  (moving  horizon)  or  outside  in 
(moving  aircraft)  frame  was  employed.  However,  the  investigators  also  examined  a  third  atti¬ 
tude  indicator,  which  was  a  side  view  display  somewhat  analogous  to  the  view  shown  in  Figure 
5c  (although  presented  as  a  moving  wing  and  fixed  horizon,  rather  than  the  moving  horizon  dis¬ 
play  shown  in  Figure  5c).  This  depiction  corresponds  in  some  respects  to  the  world-referenced 
attitude  indicator  examined  by  Haworth  and  Seery  (1992),  in  that  both  depict  vertical  (rather 
than  rotational)  movement  on  the  display,  as  the  pilot  views  a  change  in  bank  while  looking  90 
degrees  off  axis. 

Consistent  with  Haworth  and  Seery’ s  finding  regarding  attitude  recovery,  Cohen  et  al.  (2001) 
also  observed  a  cost  to  this  world  referencing  of  attitude,  relative  to  the  screen  (or  aircraft)  refer¬ 
encing  condition.  In  contrast  to  the  experiment  of  Haworth  and  Seery  (1992),  this  study  found 
no  benefit  at  all  for  the  world-referenced  display.  But,  correspondingly,  the  pilots  in  Cohen  et 
al.’s  studies  (2001)  flew  entirely  in  an  instrument  environment  with  no  outside  view,  so  that 
there  was  no  source  of  perceptual  conflict  with  the  outside  world,  and  no  possible  benefit  to 
scene  linking  of  the  outside  world  in  attention  division.  Such  a  conflict  might  otherwise  have 
been  observed  in  a  side  view,  screen-referenced  attitude  display  and  indeed  this  conflict  imposed 
the  penalties  observed  by  Haworth  and  Seery  (1992). 

Display  Symbology 

Attitude  displays  (head-down,  head-up,  and  helmet-moimted)  have  symbology  to  help  recognize 
unusual  or  imdesirable  attitudes  that  usually  result  from  SD  (cf ,  Figure  6).  Such  symbology 
should  be  enhanced  based  upon  the  latest  research  results  (Previc  &  Ercoline,  1999;  Meehan, 
2001).  In  particular,  the  impact  of  NVGs  is  ripe  for  further  research  and  how  to  accommodate 
the  visual  limitations  caused  by  NVGs  (Berkley  &  Martin,  2000). 


Figure  6.  Military  standard  HUD  symbology. 


Recent  research  results  (e.g.,  Bainbridge,  1999;  ISO,  2002)  suggest  the  development  and  testing 
of  a  display  icon  that  uniquely  conveys  the  existence  of  an  unusual  attitude  and  what  to  do  about 
it.  We  propose  that  this  icon  conform  to  characteristics  that  make  it  easy  to  detect,  interpret,  and 
act  upon  without  confusion  or  control  reversal  errors  (CREs).  Such  a  display  icon  should  be  a 
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unique  simple  geometric  shape  that  portrays  the  relevant  information  in  an  easily  understood 
format  (Bainbridge,  1999). 

Using  the  principles  espoused  by  Bainbridge  (1999)  and  our  earlier  modeling  criteria,  the  icon 
should  portray  pitch  and  bank  in  a  way  that  makes  it  clear  what  the  attitude  is,  and  what  pilot 
must  do  to  recover  from  an  unusual  attitude.  Figure  7  illustrates  our  concept  for  an  icon  to  be 
displayed  on  head-down  displays  (HDDs),  HUDs,  or  HMDs.  Previc’s  (1998)  theory  of  visual 
fields  suggests  that  the  ideal  location  is  in  upper-right  quadrant  of  any  display. 


Advances  in  panoramic  HDDs,  HUD  symbology,  and  HMDs  beg  the  question  of  how  to  best  use 
each  of  these  displays  to  enhance  pilot  attitude  awareness  and  to  combat  SD.  Ercoline,  a  leading 
SD  researcher,  has  challenged  the  need  for  HMD  attitude  symbology,  and  suggested  a  study 
using  4  experimental  objectives:  agree  on  symbology  across  the  services;  agree  on  flight  tasks; 
develop  a  protocol;  and,  establish  research  facilities  and  staff  (Ercoline,  1998).  The  point  is  that 
there  is  not  an  expert  consensus  about  whether  or  how  to  use  HMD  symbology  to  prevent  SD. 

Pilots  maititain  attitude  awareness  via  their  visual  scan;  but,  when  SD  occurs,  the  typical  feeling 
is  of  a  conflict  between  their  vision  and  somatic  senses.  It  feels  like  their  vision  is  lying  to  them. 
Therefore,  relying  primarily  on  visual  display  cues  to  recover  attitude  awareness  is  probably  less 
effective  than  using  a  combination  of  multisensory  displays.  Recent  research  has  explored  using 
auditory  and  tactile  senses  to  combat  SD.  This  recent  research  is  laudable  precisely  because  SD 
accident  rates  have  remained  constant  during  the  decades  when  attitude  information  appeared 
only  on  visual  displays.  Clearly,  a  new  emphasis  is  needed.  We  next  examine  auditory,  then 
tactile  countermeasures  for  SD. 

Auditory  Countermeasures 

A  key  feature  highlighted  in  our  previous  discussion  of  visual  display  countermeasures,  was  the 
ability  of  displays  to  capitalize  upon  parallel  processing,  or  separate  resources,  so  that,  for  exam¬ 
ple,  ambiait  vision  could  process  a  peripheral  horizon,  even  as  focal  vision  was  involved  with 
other  aspects  of  visual  perception  (Weinstein  &  Wickens,  1992;  Previc,  1998).  Correspondingly, 
a  long  history  of  research  on  multiple  processing  resources  (Wickens,  2002),  has  revealed  that 
the  auditory  modality  can  process  information  in  parallel  with  the  visual,  and  therefore  the  audi¬ 
tory  channel  should  be  able  to  support  spatial  orientation  in  an  otherwise  visually  loaded,  or 
visually  impaired  (e.g.,  by  nystagmus)  environment.  In  making  this  suggestion  however,  it 
should  be  borne  in  mind  that  the  auditory  modality  is  inherently  less  compatible  for  providing 
spatial  information,  than  is  the  visual  (Wickens,  Sundry,  &  Vidulich,  1983). 
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Potentially  useful  auditory  information  can  be  classified  into  four  categories;  the  first  two  are 
discrete/symbolic,  and  the  second  two  are  continuous/analog. 

(a)  Discrete  alerts.  This  describes  the  standard  advantage  of  the  omnidirectional,  and  intru¬ 
sive  auditory  modality  for  presenting  critical  warnings.  A  sharp  sound  or  single  carefully 
chosen  word  can  warn  pilots  that  they  may  be  experiencing  SD. 

(b)  Verbal  commands.  Verbal  commands  can  provide  effective  corrections  about  what  to  do 
to  recover  from  an  SD  incident,  as  the  verbal  modality  is  well  suited  for  command  dis¬ 
plays  (Wickens  &  Hollands,  2000),  and  there  are  well-documented  successes  in  aviation 
for  such  commands,  such  as  the  “climb  climb”  resolution  advisory  in  TCAS,  or  the  “pull 
up”  command  of  the  Ground  Proximity  Warning  System  (see  Wickens,  2003  for  a 
review).  In  both  cases,  the  commanded  verbs  directly  signal  an  action,  and  the  verbal 
modality  can  break  into  a  visually  satmated  processing  stream.  It  is  easy  to  envision  how 
a  simple  verbal  command  of  the  correction  required  to  pull  out  of  an  unwanted  attitude 
could  be  effective,  (e.g.,  “bank  left”),  assuming  that  such  a  command  was  offered 
following  the  correct  inference  of  aircraft  state. 

(c)  Continuous  localization.  A  fairly  extensive  body  of  research  has  established  the  ability  of 
auditory  tones  or  sounds  to  convey  information  as  to  the  location  of  their  source,  by 
mimicking  the  central  and  peripheral  acoustic  effects  of  the  two  ears  (Nelson  et  al.,  1998; 
Begault  &  Pittman,  1996).  Such  systems  typically  employ  intensity  and  phase  differ¬ 
ences  between  the  two  dichotically  presented  sounds,  to  accurately  signal  the  azimuth  of 
a  perceived  sound,  relative  to  the  momentary  orientation  of  the  head.  A  very  naturalistic 
or  “ecological”  signal  of  head  orientation  relative  to  the  source  of  sound  can  be  provided, 
offering  good  support  in  airborne  target  acquisition  studies  (e.g.,  Begault  &  Pittman, 

1996;  Nelson  et  al.,  1998;  Endsley  &  Rosiles,  1995).  However,  an  unfortunate  aspect  of 
such  systems  is  that  they  are  far  less  accurate  in  conveying  elevation  information  than 
azimuth  information.  Fiuthermore,  spatial  disorientation,  as  we  have  noted,  is  attributed 
to  a  failure  to  appreciate  which  direction  is  up,  rather  than  which  direction  is  forward. 
Hence  auditory  localization  is  less  perfectly  suited  for  orientation  than  for  location. 

(d)  Continuous  orientation.  To  provide  a  continuous  signal  of  which  way  is  up  typically 
requires  some  non-naturalistic  assignment  of  head  orientation  to  tone  pitch  and  or  inten¬ 
sity.  In  this  regard,  effective  demonstrations  of  auditory  compensatory  tracking  have 
been  provided  by  various  researchers  (Forbes,  1946;  Vinje  &  Pitkin,  1972;  Costello, 

1976;  see  Wickens,  1986  for  a  review).  Typical  is  the  assignment  made  by  Vinje  and 
Pitkin,  in  which  the  ear  in  which  a  tone  was  presented  represented  the  side  of  an  error  (in 
this  case,  an  attitude  displacement  from  level),  while  the  pitch  of  the  tone  represented  the 
magnitude  of  the  error.  Such  a  system  would  create  an  increasing  pitch  tone,  as  the  air¬ 
craft  rolls  to  one  side  or  the  other.  It  has  been  found  fairly  effective  in  supporting  con¬ 
tinuous  compensatory  tracking,  although  greater  benefits  are  often  realized  when  the 
auditory  cues  are  offered  redundantly  with  visual  cues,  than  in  a  single  task  auditory 
environment. 
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An  important  feature  in  designing  such  systems  for  SD  applications  is  that  the  pilot  can  gain 
greater  information  from  the  time-varying  trend  of  tones  and  sounds,  than  from  the  absolute 
pitch  or  intensity  levels  of  such  sounds.  For  example,  an  increasing  pitch  (or  volume)  can  signal 
that  things  are  getting  worse.  But  there  is  no  particular  level  of  pitch  (or  volume)  that  can  be 
used  to  intuitively  convey  a  particular  state  (e.g.,  middle  C,  or  70  decibels  corresponds  to  45°  of 
bank).  A  second  issue  is  whether  the  direction  of  a  tone  (e.g.,  left  or  right  ear)  should  correspond 
to  the  status  of  error,  or  the  necessary  command  to  correct  the  error.  Will  a  rising  tone  in  the 
right  ear  intuitively  signal  that  the  right  wing  is  rising  (and  therefore  the  plane  is  banking  to  the 
left),  or  the  converse?  Such  a  mapping  of  tone  to  state  must  be  done  consistently  and  intuitively 
by  the  pilot,  otherwise  control  reversals  will  result  (as  has  been  observed  for  the  visual  moving 
horizon  display  being  confusing,  as  discussed  above).  The  importance  of  this  intuitive  mapping 
is  heightened  for  the  SD  application,  because  the  usage  of  any  auditory  cuing  will  be  required 
when  the  pilot  may  already  be  in  a  stressed  state,  in  which  non-intuitive  mappings  are  likely  to 
be  ignored  or  misinterpreted  (Wickens,  1995). 

In  summary,  audio  tones  are  good  as  alerts  of  undesirable  states,  and  audio  commands  are  effec¬ 
tive  in  directing  recoveries  from  unusual  attitudes;  but  tones  are  not  intuitive  for  orientation 
cueing. 

Tactile  Displays 

Early  research  into  tactile  displays  was  by  Gilson  et  al.  (1975),  who  described  a  lever  system 
attached  to  the  control  yoke.  Their  goal  was  to  indicate  AOA  to  pilots  via  a  lever  that  would 
push  into  the  pilot’s  hands  to  indicate  which  direction  the  pilot  should  move  the  yoke  (push  or 
pull)  for  the  optimal  AOA  during  low  speed  flight  phases  (i.e.,  takeoff,  approach,  landing). 
Solomonow  et  al.  in  1977  and  1978  examined  human  sensitivity  to  vibrotactile  displays,  the 
early  forerunners  of  today’s  tactile  vests.  Their  goal  was  to  determine  appropriate  vibration  fre¬ 
quency  ranges,  pulse  widths,  and  body  locations  for  maximum  sensitivity  and  information  trans¬ 
fer.  Kaczmarek  (2000)  examined  adaptation  and  stimulus  threshold  changes  with  repeated 
vibrotactile  stimuli. 

Recent  developments  include  full  tactile  vests,  used  to  transfer  altitude  deviation  and  attitude 
information  (Rupert,  2000;  Raj  et  al,  2000;  McGrath,  2000;  Rochlis  &  Newman,  2000;  Spence, 
2002;  van  Erp,  2003,  Engineering  Acoustics,  Inc,,  2003).  Tactile  systems  are  typically  used  to 
complement  visual  or  audio  displays,  not  in  isolation,  because  of  their  lack  of  comparable  sensi¬ 
tivity.  Research  into  this  promising  display  technology  is  ongoing  due  to  its  relative  ease  of 
implementation,  and  early  results. 

In  many  respects  the  creation  of  tactile  displays  can  match  applications  a,  c,  and  d  of  auditory 
displays  discussed  in  the  previous  section,  and  there  is,  indeed  a  fairly  close  correspondence 
between  the  two  modaUties.  Tactile  stimulation  can  (a)  alert  (Sklar  &  Sarter,  1999),  it  can  (c) 
convey  azimuth  location  information  (for  example,  a  stimulus  presented  from  a  belt  around  the 
waist),  and  some  sense  (d)  of  orientation  (e.g.,  differential  elevation  between  the  left  and  right 
sides).  Research  on  tactile  channels  for  orientation  however  is  considerably  more  sparse  than  for 
auditory  channels  (Rupert,  2000;  van  Erp,  2003),  Both  modalities  would  probably  suffer  from 
habituation,  and  so  must  be  used  sparingly  and  in  concert  with  other  sensory  inputs. 


24 


F33615-03-M-6360 


Multisensory  Integration  for  Pilot  Spatial  Orientation 


Display  Technologies  Summary 

Our  previous  discussion  has  focused  on  visual  displays,  because  of  the  precise  spatial  informa¬ 
tion  that  these  can  provide.  However  both  auditory  and  tactile  displays  offer  potential  in  com¬ 
bating  SD  (or  maintaining  spatial  orientation),  because  of  their  capability  of  providing  informa¬ 
tion  in  parallel  with  an  overloaded  visual  system  (Wickens,  2002).  Indeed  auditory  localization 
displays  have  provided  useful  spatial  information  in  orienting  toward  targets  in  3D  space  (e.g., 
Begault  &  Pittman,  1996;  Nelson  et  al.,  1998),  as  well  as  in  continuous  tracking  (Vinje  &  Pitkin, 
1972;  Forbes,  1946;  see  Wickens,  1986  for  a  summary).  More  recently  tactile  and  haptic  dis¬ 
plays  have  been  explored  for  similar  purposes  (e.g.,  Raj,  Kass,  &  Perry,  2000;  van  Erp,  2003). 
While  it  is  clear  that  information  from  these  displays  can  be  processed  in  parallel  with  visual 
processing,  thereby  potentially  exploiting  multiple  resources,  it  remains  to  be  seen  the  extent  to 
which  such  parallel  processing  will  take  place  in  the  high  stress  and  workload  of  spatial 
disorientation. 

New  technologies,  such  as  3D  audio  and  tactile  displays  can  enhance  recognition  by  using  non¬ 
visual  channels  of  cognition  to  “get  through  to”  the  unaware  pilot.  Since  SD  is  often  triggered 
by  a  disparity  between  visual  and  vestibular  senses,  it  is  wise  to  stimulate  other  senses  to  aid  rec¬ 
ognition.  Audio  is  an  especially  compelling  input  because  it  tends  to  override  other  senses  for 
attention  (Wickens,  2002).  In  addition  to  the  above,  we  can  also  consider  verbal  auditory  infor¬ 
mation,  such  as  commands  to  the  pilot  on  how  a  control  should  be  moved,  or  what  instruments 
should  be  consulted,  in  order  to  restore  spatial  orientation.  These  too  are  capable  of  exploiting 
the  multiple  perceptual  resources  of  a  visually  overloaded  (or  incapacitated)  pilot. 

Visual  and  tactile  display  enhancements  may  be  less  effective  during  G-induced  SD  because  of 
vision  tunneling  and  somatic  G-force  effects  (Rochlis  &  Newman,  2000).  Researchers  must  also 
consider  how  to  integrate  new  display  ideas  with  existing  cockpit  displays,  audio,  etc.  The  goal 
is  to  use  the  “best”  sensory  cues  for  the  particular  SD  conditions,  if  the  event  characteristics  can 
be  confidently  established. 

In  conclusion,  it  would  appear  that  a  number  of  features  in  both  visual  and  non-visual  displays 
can  support  the  pilots’  perception  of  attitude.  For  visual  displays,  many  of  these  capitalize  upon 
ambient  vision  and  the  presentation  of  attitude  across  a  wide  range  of  visual  angle.  For  non¬ 
visual  displays,  they  exploit  the  multiple  sensory  resources  of  the  pilot.  Given  other  findings  in 
the  research  literature  supporting  the  advantage  of  redundant  presentation  of  information 
(Wickens  &  Hollands,  2000),  it  would  appear  that  offering  attitude  information  through  redun¬ 
dant  channels  could  be  a  valuable  tool  to  restoring  spatial  orientation.  In  the  following  section, 
we  suggest  ways  in  which  this  might  be  accomplished. 

intelligent  Aiding  Systems 

Intelligent  onboard  systems  that  can  recognize  SD  and  its  causes  would  provide  a  huge  advan¬ 
tage  in  the  fight  against  negative  SD  consequences.  Such  systems  would  not  only  tailor  displays 
to  the  particular  SD  event,  but  could  trigger  recovery  actions,  such  as  synthetic  speech  to  talk  the 
pilot  tluough  the  appropriate  recovery.^  An  even  more  intelligent  system  could  also  initiate 
recovery  without  jeopardizing  the  mission  by  considering  such  factors  as  position  relative  to 

Personal  communication  with  Dr.  Kristen  Liggett  about  such  an  experiment  (1 1/21/02):  Audio  recovery 
instructions  seemed  to  be  effective  and  pilots  liked  it;  results  not  yet  published. 
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enemy  airspace,  threat  warning  system  inputs,  and  fire  control  modes.  Triggering  an  auto¬ 
recovery  without  considering  mission  impact  and  pilot  intent  is  unacceptable,  except  in  the  most 
extreme  circumstances.  Auto-recovery  exists  on  the  Swedish  Grippen  in  the  form  of  auto-GCAS 
(Scott,  1999).  However,  US  pilots  distrust  auto-recovery  systems  due  to  potential  errors  and  the 
possibility  of  adversely  affecting  the  mission.”  Because  of  the  drawbacks  of  a  fully  automated 
system,  a  pilot-activated  system  could  be  an  interim  step.  However,  a  pilot-activated  recovery 
system  would  not  be  effective  for  Types  1  or  3  SD,  by  definition.  Manual  activation  would  be 
effective  for  Type  2  SD  events  and  so  should  be  a  part  of  the  layered  solution,  at  least  until  its 
efficacy  can  be  proved  or  disproved. 

Extensive  research  has  been  done  on  aircraft  state,  pilot  intent,  and  h^ard  monitoring  intelligent 
systems.*^  Such  systems  would  be  relatively  easier  to  implement  for  civilian  than  for  military 
aircraft,  and  so  can  be  developed,  tested,  and  certified  in  the  more  benign  civil  environment  as  a 
prelude  to  military  systems,  if  warranted. 

Most  importantly,  an  intelligent  cockpit  system  would  trigger  the  SD  countermeasure(s)  deter¬ 
mined  to  be  most  effective  in  any  given  situation.  Due  to  the  likelihood  of  sensory  overload,  an 
intelligent  SD  Aiding  System  would  apply  multisensory  coimtermeasures  for  redundancy, 
urgency,  and  for  improving  the  odds  of  “getting  tiirough”  to  the  pilot.  We  have  already  dis¬ 
cussed  visual,  auditory,  and  tactile  displays;  one  other  sensory  channel  remains:  olfactoiy.  An 
intelligent  system  could  trigger  a  unique  harmless  odor  (similar  to  a  unique  audio  tone)  that 
means,  “Spatial  Disorientation  strongly  suspected;  check  your  instraments!”  For  incapacitated 
or  unconscious  pilots  (Type  3  SD),  an  odor  similar  to  smelling  salts  could  help  the  pilot  regain 
consciousness  and  control. 

Post-Flight  Analyses 

All  the  old  and  new  technologies  to  combat  SD  incidents  have  uncertain  effectiveness,  unless  a 
thorough  data  gathering  and  analysis  effort  is  begun.  The  goal  should  be  to  better  understand 
and  characterize  SD  events,  and  to  determine  the  efficacy  of  the  various  SD  mitigation  tools. 
Only  with  post  hoc  (post  SD  event)  analyses  will  researchers  have  an  objective  method  to  assess 
file  most  effective  SO  enhancing,  or  SD  prevention  and  recovery  techniques  devised  by  research¬ 
ers  (Lyons  et  al.,  1993).  Also,  an  operational  analysis  system  provides  the  feedback  and  statis¬ 
tics  needed  to  validate  laboratory  research  results.  And,  data  analyses  provide  an  objective  foun¬ 
dation  for  developing  and  testing  human  performance  models,  such  as  the  multisensory  models 
described  in  the  next  chapter. 

Recent  post-flight  data  analysis  efforts  in  worldwide  commercial  aviation  have  paid  huge  divi¬ 
dends  in  understanding  the  impact  of  changes  to  such  key  flight  safety  components  as  airplane 
systems,  pilot  training,  air  traffic  control  procedures,  and  airport  conditions.  The  US  Air  Force 
has  recently  begun  such  a  fli^t  data  monitoring  program  for  its  C-17s.  The  value  of  regular 
consistent  objective  feedback  is  enormous,  as  global  aviation  data  analysis  and  sharing  programs 
attest. An  analogous  effort  for  SD  research  is  absolutely  vital. 


**  Personal  communication  with  Dr.  Kristen  Liggett  about  auto-recovery  systems  (11/21/02). 

C.f.,  Pilot’s  Associate,  Rotorcraft  Pilot’s  Associate,  French  Electronic  Cbpilot,  German  Cockpit  Assistant. 

Such  programs  are  known  as  FOQA  and  MFOQA  in  the  US,  and  FDM  in  Europe  and  Asia.  The  global  aviation 
date  sharing  program  is  GAIN  (see  www.gainweb.org). 
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Conclusion 

There  are  many  known  SD  illusions  and  causes.  How  to  best  combat  each  one  is  the  challenge 
of  attitude  awareness  and  SD  research.  The  best  solution  is  to  blend  all  of  the  known  helpful 
training,  technologies,  and  techniques,  and  to  tailor  each  to  the  particular  SD  circumstances. 
Researchers  are  investigating  a  wide  variety  of  technologies.  Visual  components  include  head- 
down,  head-up,  and  helmet-mounted  displays.  Audio  components  include  3D  audio  tones  and 
verbal  recovery  guidance.  Tactile  components  include  vibrating  vests.  Olfactory  cues  might  be 
useful  when  the  pilot’s  other  senses  are  overloaded,  or  when  he  or  she  is  incapacitated.  Training 
includes  classroom  lectures,  centrifuge  demonstrations,  and  practice  flights.  Post-flight  data 
analyses  “close  the  loop”  in  the  research  approach  by  providing  feedback  about  which  tech¬ 
niques  are  the  most  effective. 

The  anticipated  benefits  of  this  research  and  development  are  to  help  pilots  maintain  spatial  ori¬ 
entation  in  flight,  or,  when  orientation  degrades,  to  recognize  and  recover  firom  spatially  disori¬ 
enting  situations,  thus  preventing  SD’s  negative  consequences.  The  cost  impact  of  SD  on  the  US 
military  is  over  $300  million  per  year,  not  counting  lives  lost.  US  civilian  losses  are  comparable. 
Devising  a  layered  approach  to  even  incrementally  improve  the  situation  has  a  multi-million 
dollar  per  year  potential  benefit. 
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IV.  Modeling 

The  goal  of  modeling  attitude  awareness,  spatial  disorientation,  and  multisensory  workload  is  to 
provide  a  framework  for  assessing  or  inferring,  in  real  time,  the  pilot’s  level  of  awareness  of  the 
current  aircraft  orientation,  and  then,  based  upon  this  assessment,  implementing  an  appropriate 
set  of  interventions  to  counter  any  assessed  spatial  disorientation  events,  and  their  consequences. 

We  first  characterize  the  pilot’s  internal  state  of  SD,  and  then  discuss  the  objective  measures  that 
are  necessary  to  infer  that  state  from  external  variables.  Lastly,  we  introduce  and  explain  a  mul¬ 
tisensory  enhancement  to  Wickens’  Multiple  Resource  Theory  (MRT)  -  something  we  call 
“SAWOY.” 

Terminology  &  Formulas 

1,  Explicit  meaning  of  attitude  awareness:  Accurate  knowledge  ofmomentary  Attitude  (A) 
relative  to  the  earth’s  frame  of  reference  (vertical).  As  discussed  in  Chapter  1  of  this  report, 
the  focus  on  attitude  is  directed  because  the  failure  of  attitude  awareness  appears  to  be 
responsible  for  nearly  all  SD  events.  Loss  of  attitude  awareness  leads  to  a  stall  condition, 
rapid  descent,  or  unwanted  turns,  both  of  the  latter  leading  to  undesirable  trajectories  (the 
latter,  in  turn  can  lead  to  geographical  disorientation  -  a  separate  issue,  not  modeled  here). 
For  purposes  of  modeling,  we  define  attitude  as  a  vector  quantity  composed  of  pitch  (P)  and 
bank  (B). 

2.  Computational  expression  of  the  loss  of  attitude  awareness;  i.e..  spatial  disorientation 

fSD):  Ideally,  we  would  like  to  have  a  bounded  expression  of  SD  varying  between  0  and  1 
(or  possibly  between -1  and +1;  see  below).  This  makes  calculations  more  tractable.  Fora 
pilot  with  perfect  attitude  awareness,  SD  =  0,  and  therefore  perceived  attitude  (Ap)  =  true 
attitude  (At). 

Given  this  expression,  then,  the  loss  of  attitude  awareness  (onset  of  SD)  can  be  expressed  in  two 
distinct  ways,  each  with  different  implications: 

(1)  SD  =  VaitAp)/90.  This  implies  that  flie  loss  of  attitude  awareness  is  related  to  the  variance 
in  (uncertainty  of)  the  momentary  attitude,  scaled  by  a  maximum  value  of  90  degrees.  Thus 
a  pilot  who  has  absolutely  no  idea  of  which  way  is  up,  would  have  SD  =  1 .0  (this  assumes 
that  the  variance  of  a  rectangular  distribution  ranging  from  0-180  is  90).  We  abbreviate  the 
first  formula  as  VAP.  In  essence,  this  is  a  confidence  measure.  Pilots  in  a  Type  n  SD  event 
know  that  they  are  disoriented.  That  is,  they  are  aware  that  they  have  little  or  no  confidence 
in  which  direction  the  true  vertical  is.  Having  no  confidence  yields  SD  =  1  for  this  formula. 

(2)  SD  =  (|Ap-Af|)/180.  This  implies  that  the  loss  of  attitude  awareness  is  a  direct  measure  of 
how  much  the  perceived  attitude  departs  from  the  true  attitude.  It  suggests  that  an  illusion 
has  caused  a  firm  belief  in  an  incorrect  attitude.  Here  SD  >  0  is  a  firm  belief  that  something 
ottier  than  vertical  is  file  upright  (and  SD  =  1 ,  for  example,  is  a  belief  that  you  are  flying 
right-side  up,  when  you  are  inverted,  or  vice  versa).  We  abbreviate  formula  2  by  DA 
(Difference  in  Attitude).  In  contrast  with  formula  1  (VAP),  there  is  no  single  positive  (incor¬ 
rect)  belief  associated  with  SD  >  0.  So,  for  example,  pilots  experiencing  the  Leans  typically 
believe  that  they  are  certain  where  the  trae  vertical  is,  but  they  are  wrong.  If  they  feel  they 
are  level  when  in  fact  they  are  banked  90  degrees,  then  SD  =  0.5  for  this  formula. 
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It  seems  apparent  that  VAP  is  associated  with  Type  2  (recognized)  SD,  and  DA  is  associated 
with  Type  1  SD.  That  is,  if  pilots  do  not  know  that  they  are  disoriented,  they  will  probably  hold 
a  strong  belief  that  something  other  than  the  true  vertical,  is  the  perceived  vertical  (large  DA). 
Knowledge  that  they  are  disoriented  however  might  simply  throw  the  pilots  into  a  state  of  confu¬ 
sion  typified  by  high  VAP. 


There  are  two  ways  of  thinking  about  DA  >  0.  As  the  formula  states,  it  is  directly  proportional  to 
the  deviation  of  perceived  attitude  fi*om  true  attitude,  so  that  an  SD  of  0.5  (e.g.,  pilots  assume 
they  are  level,  when  they  are  banked  90  degrees)  is  half  as  bad  as  an  SD  of  1 .0  (e.g.,  pilots 
assume  that  they  are  level,  when  in  fact  they  are  inverted).  This  measure  does  not  then  distin¬ 
guish  the  degree  of  certainty  in  the  incorrect  attitude.  It  would  seem  that  a  pilot  who  is  certain  of 
a  90-degree  offset  (DA  =  0.5),  has  greater  SD  than  one  who  believes  (but  is  uncertain)  that  he  or 
she  may  be  at  a  1 80-degree  offset  firom  the  true  attitude  (calculated  DA  =  1 .0).  This  greater 
imcertainty  is  reflected  in  a  higher  VAP.  Thus  we  can  combine  the  two  concepts  VAP  and  DA 
(Figiue  8),  such  that  larger  penalties  are  applied  for  smaller  VAPs  at  larger  DA  (i.e.,  “strong  but 
wrong”  is  worse  than  “weak  but  wrong”). 


Worst  Case 


Severity 
of  Loss 


Figure  8.  Variance  of  perceived  attitude  (VAP),  and  the  difference  (DA)  between  perceived  attitude  (Ap) 
and  true  attitude  (At)  in  relation  to  the  loss  of  attitude  awareness  (or  onset  of  SD). 


Finally,  we  recall  that  A,  the  vector,  is  composed  of  P  and  B.  It  may  be  that  Pp  and  Bp  (per¬ 
ceived  pitch  and  bank)  are  perfectly  correlated  across  all  illusions  and  distortions,  but  indeed  this 
is  probably  not  the  case.  For  example,  the  somatogravic  illusion  in  which  acceleration  leads  to 
perceived  pitch  up  (or  deceleration  leads  to  perceived  pitch  down)  certainly  affects  Pp,  but  not 
Bp.  The  Leans  as  a  precursor  to  the  graveyard  spin,  effects  Bp  initially,  but  not  Pp.  Hence  an 
open  issue  is  whether  to  compute  a  single  vector  of  Attitude,  or  to  break  it  into  its  components  of 
P  and  B.  In  the  latter  case,  SD  scales  from  0  to  2. 

Our  discussion  above  has  focused  on  the  actual  state  of  SD,  which  exists  within  the  mind  of  the 
pilot.  An  intelligent  SD  Aiding  system  will  not  have  direct  access  to  that  state,  but  must  infer  it 
on  the  basis  of  available,  measurable  signals,  which  may  be  combined  computationally.  We 
consider  this  computation  as  follows: 
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3.  Computational  basis  of  SD:  It  is  feasible  to  think  of  SD  =  0  (or  attitude  awareness  =  1)  as 
defined  by  a  state  in  which  the  pilot,  in  VMC,  is  giving  fiill  attention  to  the  visual  horizon, 
which  always  (except  at  very  high  altitudes)  provides  an  accurate  representation  of  the  verti¬ 
cal.  As  such,  from  this  state,  there  are  a  series  of  factors  that  can  drive  SD  upward,  some  of 
which  have  a  quantitative  basis  in  prior  validating  modeling  efforts.  Among  these  are: 

3.1  Reduced  quality  of  visual  input.  Use  of  the  attitude  indicator  (e.g.,  at  night  or  in  IMC), 
requires  focal  and  foveal  vision,  which  is  less  effective  in  providing  intuitive  ambient 
attitude  information  (Previc,  1998),  As  the  AI  is  diminished  in  size,  or  more  ambiguous 
about  what  is  represented  by  the  moving  element  (Roscoe,  1968),  the  quality  of  correct 
attitude  information  is  reduced.  As  the  ambient  quality  of  this  information  is  improved, 
for  example  by  use  of  a  Malcomb  horizon,  or  background  AI  (Liggett  et  al.,  1999),  the 
contributions  of  the  AI  improve  to  approach  that  of  the  true  horizon  (SO  =  1 ,  SD  =  0). 

3.2  Reduction  of  attention  to  true  horizon.  There  are  a  variety  of  computational  models 

that  can  characterize  the  loss  of  precision  of  spatial  estimation  associated  with  attention 
diversion.  Most  appropriate  may  be  the  optimal  control  model  of  Levison  and  colleagues 
(1971)  that  associates  an  increase  in  variance  of  a  perceived  quantity,  as  to  be  inversely 
proportional  to  the  diversion  of  attention.  As  such  this  can  be  represented  as  a  contribu¬ 
tion  of  attention  diversion  to  VAP  (VAP  =  1  -  attn,  where  “attn”  is  modeled  as  the  pro¬ 
portion  of  attention  allocated  to  attitude  information;  often  measured  by  scanning 
(Wickens  et  al.,  2003).  This  formula  does  not  distinguish  where  attention  is  diverted  to, 
and  could  equally  describe  attention  divated  to  an  auditory  chaimel,  as  to  a  visual  source 
in  tile  cockpit  (other  than  the  AI).  For  attention  diverted  to  other  visual  channels,  scan¬ 
ning  measures  could  be  used  (Wickens  et  al.,  2003).  For  attention  diverted  to  auditory 
chaimels,  inferences  must  be  made  (e.g.,  the  duration  of  diversion  is  related  to  the  length 
of  an  auditory  communication  in  the  cockpit,  to  which  the  pilot  responds). 

It  is  also  possible  to  construct  the  model  such  that  the  diversion  of  attention  away  from  either  the 
true  horizon,  or  the  attitude  indicator,  will  not  only  increase  the  variance  of  perceived  attitude 
(VAP),  but  also  provide  an  opportunity  for  attentional  capture  by  the  insidious  vestibular 
system,  to  the  extent  that  the  latter  provides  erroneous  cues.  This,  in  turn,  would  create  an 
opportunity  for  DA.  The  advantage  of  this  approach  is  that  the  misleading  signals  provided  by 
the  vestibular  system  can  be  directly  calculated  (and  therefore  are  amenable  to  computational 
modeling),  given  the  known  properties  of  the  otoliths  and  semi-circular  canals  to  provide  false 
information  during  periods  of  sustained  accelerations.  The  amount  of  such  signals  can  range 
from  slight  (e.g.,  the  Leans,  in  a  sustained  turn)  to  large  (Coriolis).  Finally  while  some  other 
channels  of  information  may  degrade  attitude  awareness  by  distraction,  other  artificial  channels 
can  restore  attitude  awareness,  even  if  they  direct  attrition  away  from  the  valid  horizon  sources 
of  correct  attitude  information.  Such  artificial  chaimels,  as  we  have  previously  noted,  would 
include  auditory  stimulation,  voice  commands,  and  tactile  sensors. 

In  summary: 

•  Attitude  awareness  can  be  modeled  with  a  “gold  standard”  of  Ap  =  At  in  VMC. 
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•  Degradation  from  this  state  (that  is,  SD)  can  be  modeled  by  reduced  quality  attitude  informa¬ 
tion  within  the  cockpit,  diversion  of  attention  away  from  attitude  sources,  and  direction  of 
attention  toward  the  vestibular  system  at  times  of  accelerated  flight. 

•  Attention  away  is  captured  by  the  measure  VAP  and  there  are  quantitative  models  existing  to 
describe  this. 

•  The  cost  of  attention  directed  toward  the  vestibular  inputs  can  be  calculated  by  physical  for¬ 
mula  related  to  the  effects  of  sustained  acceleration  on  the  otoliths  and  semi-circular  canals, 
which  yield  misleading  signals.  The  latter  constitutes  a  separate,  independent  module  of  the 
computational  model,  and  can  be  used  to  predict  DA. 

Simulation  experiments  in  Phase  II  will  be  designed  to  assess  the  extent  to  which  the  degree  of 
SD  actually  experienced  by  pilots  will  correlate  with  the  predicted  variables  described  above. 

A  final  property  of  any  computational  model  to  be  used  in  implementing  countermeasures  is  the 
model’s  own  assessment  of  self-confidence  in  its  assessment  of  SD  state.  One  can  envision  two 
scenarios,  both  with  the  same  assessment  of  SD  level  (e.g.,  VAP  =  0.9),  but  one  assessment 
reached  on  the  basis  of  a  number  of  very  reliable  information  channels,  and  the  other  reached  on 
the  basis  of  a  small  number  of  unreliable  sources.  Aii  intelligent  system  should  be  considerably 
more  cautious  in  intervening  aggressively  (e.g.,  taking  control  of  the  aircraft)  in  the  latter  case 
than  in  the  former.  Where  possible,  our  approach  provides  some  measure  of  the  confidence  in 
the  SD  assessment. 

Because  attention  mismanagement  (i.e.,  distraction)  is  a  key  component  in  many  SD  mishaps  and 
beeause  it  is  important  to  consider  in  the  above  formulas  (VAP  and  DA),  we  now  turn  to  model¬ 
ing  the  pilot’s  attention  in  the  form  of  a  multisensory  workload  measure,  S  AVVOY,  that  expands 
upon  Wickens’  Multiple  Resource  Theory. 


Expanding  upon  MRT 

Wickens’  multiple  resource  theory  (MRT)  says  to  use  the  maximum  number  of  different  sensory 
resources/channels  to  “get  through”  any  distractions.  Since  our  goal  is  a  multisensory  approach 
to  combating  SD  and  since  Wickens’  MRT  focuses  on  visual  and  auditory  channels,  we  have 
expanded  MRT  to  include  somatic,  vestibular,  and  olfactory  senses.  The  resulting  workload 
model,  which  we  call  SAWOY  (somatic,  auditory,  visual,  vestibular,  olfactory,  psychomotor¬ 
primary,  psychomotor-secondary,  and  cognitive),  is  useful  for  helping  our  SD  Aiding  System 
(detailed  in  Chapter  VI)  determine  which  countermeasure(s)  to  apply  under  varying  conditions. 
For  example,  if  the  pilot  is  in  a  high  G  maneuver,  tactile  cues  are  less  likely  to  be  detected  by  the 
pilot  due  to  “high  somatic  workload.”  Similarly,  if  the  pilot  is  talking  on  the  radio  or  to  a  back- 
seater,  audio  cues  may  be  ineffective  due  to  high  audio  workload. 

During  our  demonstration  scenarios,  we  model  the  hypothetical  pilot  workload  in  all  the 
SAWOY  channels  and  use  that  information  to  influence  the  countermeasures  presented  by  our 
SD  Aiding  System. 
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SD  Assessment  Models 

To  this  point,  this  chapter  has  presented  the  detailed  theoretical  bms  for  modeling  SD  and  atti¬ 
tude  awareness  based  oh  the  research  in  the  SD  literature.  The  remainder  of  this  chapter  is 
devoted  to  the  actual  SD  modeling  done  for  the  Phase  I  demonstration.  Two  models  of  SD  were 
developed  to  detect  the  occurrence  of  the  Leans  and  Coriolis  illusions  -  one  model  for  each  illu¬ 
sion.  Each  of  the  models  is  based  on  its  description  from  Chapter  H.  They  each  use  observable 
data  from  the  aircraft  and  pilot  to  assess  the  state  of  the  vestibular  system  relative  to  the  actual 
attitude  of  the  aircraft. 

Model  of  Leans 

The  model  of  the  Leans  is  expressed  as  a  timed  sequence  of  events  with  the  certainty  of  the 
assessment  of  the  disorientation  increasing  with  each  successive  event  as  shown  in  Figure  9.  The 
first  event  is  the  initiation  of  a  roll  at  a  rate  below  the  vestibular  threshold  (Mulder’s  constant  of 
27sec).  The  second  event  is  a  roll  angle  of  greater  than  5  degrees  that  lasts  longer  than  5  sec¬ 
onds.  If  these  two  events  occur  in  sequence,  it  is  possible  that  the  pilot  has  not  noticed  the 
ensuing  roll  angle  and  that  there  is  a  difference  between  the  pilot’s  perceived  attitude  (Ap)  and 
the  true  attitude  (At)  of  the  aircraft.  As  such,  the  model  indicates  a  possibility  of  SD  but  only  at 
a  very  low  confidence  level.  The  third  event  is  the  loss  of  altitude  as  measured  by  negative  ver¬ 
tical  velocity.  If  this  event  follows  ftie  first  two,  it  is  possible  that  the  pilot  has  also  not  noticed 
the  loss  of  sJtitude  and  the  model  represents  an  increased  confidence  in  its  ass^sment  of  the 
Leans  (shown  as  increasing  certainty  of  SD  in  Figure  9).  The  fourth  event  is  a  roll  well  above 
the  vestibular  threshold  (i.e.,  greater  than  57sec  and  with  sufficient  duration)  in  the  opposite 
direction  from  that  of  the  first  event.  If  this  occurs  following  the  first  three  events,  it  is  possible 
that  the  pilot  has  now  noticed  the  roll  angle  and  has  quickly  corrected  back  toward  level.  When 
this  occurs,  the  pilot’s  vestibular  system  will  register  a  roll  in  the  opposite  direction,  again 
resulting  in  a  difference  between  the  perceived  attitude  (Ap)  and  the  actual  attitude  (At)  of  the 
aircraft.  At  this  point  the  model  represents  a  high  level  of  SD  certainty.  The  final  event  in  tiie 
model  is  the  tilt  of  the  pilot’s  head  opposite  the  perceived  roll  angle,  ffi  this  occurs  following  the 
other  four  events,  it  is  likely  that  the  pilot  is  experiencing  the  Leans  and  the  model  represents  an 
even  higher  level  of  SD  certainty. 
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Model  of  Coriolis 

Unlike  the  Leans,  the  model  of  the  Coriolis  illusion  is  not  expressed  as  a  sequence  of  events  but 
as  an  instantaneous  occurrence.  The  model  is  expressed  by  the  maneuvering  of  the  aircraft  rela¬ 
tive  to  the  movements  of  the  pilot’s  head  in  the  three  different  planes  (yaw,  pitch  and  roll).  The 
first  event  is  the  aircraft  maneuvering  such  that  the  rate  of  change  in  any  of  the  three  planes  is 
above  the  vestibular  threshold  (Mulder’s  constant).  If  during  this  maneuvering  the  pilot’s  head 
moves  greater  than  30  degrees  from  center  in  either  of  the  other  two  planes,  then  there  is  the 
potential  for  the  Coriolis  illusion  to  occur.  For  example,  Coriolis  is  likely  to  occur  if  the  aircraft 
is  in  an  above  threshold  roll  during  which  the  pilot’s  head  moves  a  large  amount  (we  used  30°)  in 
either  the  yaw  or  pitch  planes.  Since  the  Coriolis  model  is  based  on  an  instantaneous  occurrence, 
there  is  no  representation  of  SD  certainty  or  confidence  level.  Rather,  the  model  simply  indi¬ 
cates  whether  or  not  Coriolis  may  be  occurring. 


Conclusion 

“All  models  are  wrong,  but  some  are  useful”  (Box,  1979). 

The  most  important  benefit  from  modeling  is  a  deeper  and  more  thorough  understanding  of  the 
problem  being  modeled.  In  starting  with  a  theoretical  approach,  we  ensured  that  we  considered 
all  of  the  relevant  aspects  of  attitude  awareness  and  spatial  disorientation.  Going  from  the  theo¬ 
retical,  to  what  we  could  practically  accomplish  in  Phase  I  with  realistic  aircraft  and  state 
parameters  meant  that  we  could  prove  the  feasibility  of  our  approach  while  remaining  cognizant 
of  enhancements  to  be  made  in  Phase  11. 

By  modeling  the  vestibular  system  and  multi-channel  workload,  we  have  learned  about  impor¬ 
tant  interactions  and  the  series  of  trigger  events  that  precipitate  an  SD  event.  In  Phase  II,  we  will 
calibrate  our  model  and  test  it  in  realistic  scenarios  with  real  pilots  in  a  motion-based  simulator. 
We  will  also  examine  work  by  Borah  et  al.  (1988)  to  see  where  we  can  leverage  their  techniques 
within  our  own  modeling  approach. 
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V.  SD  Scenarios 

To  demonstrate  the  efficacy  and  broad  applicability  of  our  modeling  and  countermeasure 
approach,  we  will  examine  both  slow-  and  rapid-onset  SD  events.  We  also  want  to  focus  on 
common  and  catastrophic  events  so  that  we  ensure  the  broad  coverage  of  our  solution.  The 
Leans  are  one  of  the  most  common  illusions  (Hohnes  et  al.,  2003),  and  so  it  is  the  focus  of  our 
first,  slow-onset  scenario.  The  compelhng  nature  of  the  Coriolis  illusion  makes  it  ideal  as  die 
focus  of  our  second,  rapid-onset  scenario. 


Scenario  #1  (slow  onset  of  SD;  the  Leans) 

On  the  third  day  of  the  war  over  Iraq,  F-22C  pilot  Captain  Luke  Keller  takes-off  with  a  foil  load 
of  bombs  for  tonight’s  target.  It’s  a  moonless  night  over  the  desert  as  Luke  climbs  to  an  inter¬ 
mediate  altitude  of  FL200.  No  need  to  go  higher,  since  the  US-led  coalition  has  air  supremacy. 
Luke  is  tired  from  the  pace  of  the  war,  but  excited  to  be  doing  real  missions  instead  of  practice 
runs  and  squadron  paperwork  back  home  at  Langley.  As  he  levels-off  and  studies  the  intelli¬ 
gence  photos  for  his  target,  the  aircraft  entere  a  very  slow  and  shallow  left  bank.  Luke  does  not 
notice  it  for  mmiy  seconds  as  his  altitude  slowly  decreases.  When  he  does,  he  snaps  back  to  his 
desired  heading  (which  induces  the  Leans).  He  feels  like  he’s  now  in  a  right  bank,  and  so  nudges 
the  stick  left.  He  oscillates  back-and-forth  with  each  cycle  diverging  into  greater  bank  angles. 
After  several  seconds,  he  is  in  a  foll-blown  unusual  attitude  and  his  altitude  is  rapidly  decreasing. 

Fortunately  for  Luke,  his  F-22C  is  equipped  with  the  latest  SD  Aiding  System,  which  detects  his 
situation  and  applies  increasingly  intrusive  countermeasures.  With  such  assistance,  Luke  recov¬ 
ers  after  what  seems  an  eternity,  but  is  only  about  60  seconds.  He  calms'  down,  climbs  back  to 
FL200  and  proceeds  on  course,  completing  his  mission  as  planned. 

When  he  returns  to  base,  he  has  quite  a  war  story  to  tell  about  how  his  SD  Aiding  System  saved  ■ 
him,  his  new  F-22C  (at  $40M  a  copy),  and  his  mission. 


Scenario  #2  (rapid  onset  of  SD;  Coriolis) 

This  scenario  is  similar  to  #1  except  that  foe  set-up  is  a  tight  turn  followed  by  a  flash  of  light 
from  an  explosion  behind  and  to  foe  right  of  Captain  Kristi  Oster.  She  quickly  turns  her  head, 
drawn  to  foe  flash,  and  experiences  a  severe  Coriolis  illusion.  Within  seconds  she  is  tumbling 
from  foe  sky  as  her  head  spins  and  her  F-15C  plummets  toward  the  desert  floor.  At  the  last 
moment,  her  newly  installed  SD  Aiding  System  triggers  an  auto-recovery  and  saves  Captain 
Oster  and  her  jet. 

After  several  minutes  to  regain  her  composure  and  contact  command  post,  she  receives  a  new 
mission  and  proceecb  on  her  way.  Upon  her  return  to  base,  she  is  met  by  the  squadron  fli^t 
surgeon  who  does  a  follow-up  exam  to  determine  if  there  are  any  lingering  effects  from  her 
incident.  The  crew  chief  and  ofoer  maintenance  folks  also  inspect  her  F-15C  for  signs  of  over¬ 
stressed  wings,  panels,  pylons,  cte. 
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Demonstration  Scenarios 

To  create  the  scenarios  for  our  demonstration,  we  used  an  off-the-shelf  PiC  flight  simulator  called 
X-Plane.^^  It  includes  a  number  of  different  aircraft  and  a  data  collection  features  that  output 
flight  data  parameters  recorded  during  a  simulated  flight.  Using  a  simulated  F-22,  the  lead 
author  flew  two  scenarios  representing  the  Leans  and  Coriolis  illusions  (as  described  in  the 
above  scenarios).  Table  3  shows  a  sample  of  the  resulting  flight  data.  The  first  column  shows 
the  elapsed  time  in  seconds  and  the  fi-equency  at  which  the  data  are  updated  (~  8  hertz).  The  two 
scenarios  are  each  about  60  seconds  long  but  the  elapsed  time  shows  them  starting  around  195 
seconds  as  it  took  that  long  to  takeoff  and  achieve  the  desired  position  within  the  simulated  flight 
before  beginning  the  desired  scenario  maneuver  sequence.  The  full  data  set  for  these  two  sce¬ 
narios  includes  600-700  data  events.  The  complete  list  of  flight  parameters  includes:  elapsed 
time,  barometric  altitude,  heading,  pitch,  roll,  pitch  rate,  roll  rate,  indicated  air  speed,  vertical 
speed  indicated,  angle  of  attack,  radio  altitude,  roll  rate  change,  pitch  rate  change,  commanded 
altitude,  and  commanded  heading. 


Table  3.  Aircraft  scenario  parameters. 


Figures  1 0  and  1 1  are  graphs  of  the  pitch  and  roll  changes  for  the  Leans  and  Coriolis  scenarios 
respectively.  The  Leans  scenario  starts  with  a  short  period  of  stable  flight  followed  by  the  initi¬ 
ating  event  of  a  slow  shallow  roll.  After  that,  the  scenario  simulates  the  stereotypical  Leans 
sequence  of  roll  oscillations  associated  with  the  confusion  between  the  pilot’s  vestibular  system 
and  the  actual  aircraft  attitude.  The  end  of  the  scenario  simulates  a  potential  recovery  fi*om  the 
erratic  control  induced  by  the  illusion. 


The  Coriolis  scenario  begins  with  a  short  period  of  stable  flight  followed  by  the  initiating  event 
of  a  hi^  roll  rate.  This  is  followed  by  the  potential  dramatic  changes  in  roll  that  could  occur 
firom  the  erratic  pilot  actions  associated  with  the  Coriolis  illusion.  A  real  occurrence  of  Coriolis 
may  also  include  the  dramatic  pitch  changes  associated  with  the  death  spiral.  However,  for  the 
purposes  of  this  demonstration,  only  erratic  roll  conunands  were  simulated.  As  with  the  Leans 
scenario,  the  Coriolis  scenario  ends  with  a  recovery. 


For  details  about  X-Plane,  see  http://www.x-plane.com/. 


35 


F33615-03-M-6360 


Multisensory  Integration  for  Pilot  Spatial  Orientation 


Figure  10.  Leans  scenario  pitch  (blue)  and  roll  (magerlta). 


Figure  11.  Coriolis  scenario  pitch  (blue)  and  roll  (magenta). 


Iq  addition  to  tiie  aircraft  parametera,  the  scenarios  included  data  specific  to  the  pilot.  Since 
these  data  were  not  available  ftom  X-plane,  they  were  created  manually.  The  data  included  pilot 
head  position  in  three  planes  ^itch,  roll  and  yaw),  and  SAWOY  workload  scores.  The  head 
positions  were  r^resaited  as  angles  and  were  added  random  numbere.  Most  of  the  SAWOY 
workload  values  were  also  added  as  random  numbers,  except  where  the  pilot  was  experiencing 
higher  G-forces.  In  these  cases,  higher  values  were  included  for  the  somatic  channel,  which 
inhibited  the  use  of  the  tactile  vest  countermeasure. 
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Conclusion 

The  importance  of  effectively  combating  SD  along  the  whole  range  of  operational  exposure  is 
absolutely  critical  to  reducing  the  human  and  aircraft  costs  of  this  major  flying  problem  and 
accident  causal  factor.  Clearly,  as  described  in  the  above  scenarios,  it  is  much  less  expensive  to 
combat  SD  than  to  replace  pilots  and  aircraft.  The  time  is  ripe  to  apply  prevention  and  counter¬ 
measure  strategies  to  SD  via  pilot  selection,  training,  mission  briefings,  intelligent  cockpit  aiding 
systems,  and  rigorous  post-event  analyses.  The  above  scenarios  and  our  demonstration  illustrate 
the  value  of  a  broad  approach  to  combating  spatial  disorientation. 
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VI.  SD  Aiding  System  Prototype 

The  overall  goal  of  the  SD  Aiding  System  is  to  achieve  a  layered  approach  to  solving  SD  prob¬ 
lems.  It  integrates  multisensory  displa)^  to  help  pilots  recognize  and  recover  from  SD  situations. 
The  SD  Aiding  System  prototype  demonstrates  what  could  be  done  in  either  a  simulator  or  target 
aircraft  to  alert  pilots  of  an  SD  situation,  and  to  help  them  recover. 

The  SD  Aiding  System  consists  of  three  components  (Figure  12).  The  first  component  is  the 
State  Table.  It  contains  state  data  for  the  pilot,  aircraft,  and  external  world,  which  it  uses  to  ana¬ 
lyze  the  current  state  of  the  aircraft,  world,  and  pilot’s  level  of  SD  (if  any).  There  is  also  the 
Countermeasure  Assessor,  which  uses  the  output  from  the  State  Table’s  assessors  to  determine 
the  most  effective  countermeasures  for  the  specific  SD  situation.  Once  the  countermeasures  are 
decided  upon,  the  appropriate  display(s)  are  activated. 


Figure  12.  Spatial  disorientation  aiding  system  architecture. 


As  an  SD  situation  evolves,  the  State  Table’s  assessors  continue  to  assess  the  state  of  flie  pilot, 
aircraft  and  world.  When  the  situation  changes,  it  informs  the  Countermeasure  Assessor,  so  that 
the  compensatory  actions  can  be  changed  accordingly.  Also,  as  a  situation  worsens  (i.e.,  its 
potential  consequences  become  more  severe),  the  processing  rate  of  the  system  could  change 
accordingly.  For  example,  during  a  less  severe  situation,  processing  mi^t  execute  at  1  Hz;  but, 
during  more  severe  situations,  execution  at  8  Hz  would  be  more  appropriate. 

State  Tables 

The  State  Table  helps  the  SD  Aiding  System  create  a  complete  picture  of  the  pilot’s  current 
situation  and  serves  as  a  communication  object  among  the  various  processes.  To  create  this  pic¬ 
ture,  the  system  must  know  something  about  the  aircraft,  pilot,  and  the  external  factors  (i.e., 
world)  that  are  influencing  the  pilot’s  decisions  and  physiology. 

Thft  first  pifine  of  this  picture  is  the  aircraft  for  which  raw  data  are  provided  by  the  aircraft  sys¬ 
tems  (or  model,  in  the  case  of  a  simulated  aircraft).  For  instance,  the  aiding  system  uses  pitch, 
bank,  and  airspeed  to  analyze  the  motion  and  position  of  the  plane.  These  raw  data  are  also  used 
to  derive  parameters  such  as  pitch  rate,  roll  rate,  and  change  of  (delta)  airspeed.  Both  raw  and 
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derived  data  are  stored  in  the  State  Table;  the  State  Table’s  assessors  and  the  Countermeasure 
Assessor  use  the  State  Table  data  (Table  4). 


Table  4.  Aircraft  state  parameters. 


Parameter 

■  ..Units'-  : 

Data  Type 

Comments 

Raw  Data 

Airspeed 

0  to  9999 

knots 

double 

Altitude 

-300  to  +60000 

feet 

AOA 

7 

7 

Heading  magnetic 

7 

degrees 

double 

Pitch 

degrees 

double 

— 

Pitch  command 

7 

7 

fore-aft  stick  position 

Radio  altitude 

-100  to  +6000 

feet 

double 

Roll 

-180  to +180 

degrees 

double 

Roll  command 

7 

7 

double 

left-right  stick  position 

Vertical  speed 

feet  per  minute 

double 

Derived  Data 

Aircraft  state 

0,  L2,3,4,  5, 6, 7,  8, 9 

None,  SD  Inducing,  Erratic  Control, 
Suspected  Unusual  Attitude,  Confirmed 
Unusual  Attitude,  Impending  Crash, 
Imminent  Crash,  Recovering,  Recovered 

Delta  airspeed 

-100  to +100 

knots/second 

double 

Change  of  airspeed  over  time. 

Duration  of  roll 

0  to  00 

seconds 

How  long  at  current  roll  angle. 

Duration  of  pitch 

0  to  00 

seconds 

How  long  at  current  pitch. 

Erratic  control 

false,  true 

Ground  impact 

0-120 

seconds 

double 

Depends  on  Vertical  speed  and  Radio 
altitude. 

Mulder’s  Pitch 

7 

7 

Mulder’s  Pitch  Exceed 
Limit 

0,1 

false,  true 

Mulder’s  Roll 

7 

7 

Mulder’s  Roll  Exceed 

Limit 

7 

7 

Near  flight  envelope  limit 

AOA  limits? 

double 

Pitch  rate 

0  to  00 

Change  of  pitch  over  time. 

Pitch  rate  duration 

0  to  00 

seconds 

Roll  rate 

?  0  to  00 

Change  of  roll  over  time. 

Roll  rate  duration 

0  to  00 

seconds 

double 

How  long  at  the  current  roll  rate. 

Unusual  attitude 

0,1,2 

no,  suspected,  confirmed 

Vertical  accel.  duration 

seconds 

double 

How  long  at  current  Gz. 

The  second  piece  of  the  picture  is  the  pilot.  The  SD  Aiding  System  must  assess  the  current  state 
of  the  pilot  before  it  can  determine  the  extent  to  which  the  pilot  is  disorientated.  The  pilot’s  state 
also  helps  determine  the  best  compensatory  actions.  For  instance,  it  is  important  for  the  aiding 
system  to  be  able  to  determine  whether  or  not  the  pilot  is  incapacitated.  To  do  this,  the  aiding 
tool  could  use  existing  sensors  to  measure  the  pilot’s  heartbeat  rate,  skin  temperature,  pupil  dila¬ 
tion  and  other  physiological  measures  of  stress  and  incapacitation.  These  pieces  of  raw  physio¬ 
logical  data  along  with  the  derived  data  are  stored  in  the  Pilot  State  Table  (Table  5). 

The  last  data  set  needed  to  finish  the  picture  of  the  pilot’s  situation  is  the  state  of  the  external 
world.  These  data  are  important  because  external  events  can  increase  the  probability  of  spatial 
disorientation,  and  influence  how  pilots  should  attempt  to  recover  from  their  disorientation. 
Examples  of  raw  data  are  the  presence  of  threats,  time  of  day,  visibility,  and  latitudeAongitude. 
These  data  can  be  used  to  determine  if  the  pilot  is  in  combat,  whether  it  is  day  or  night,  and  the 
type  of  terrain  over  which  the  pilot  is  flying  (Table  6). 
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Table  5.  Pilot  state  parameters. 


Parameter 

Range 

Vnits 

Data  Type 

Otmitiente 

Raw  Date 

? 

? 

Somatic  workload 

OikeVACP) 

double 

0-7 

(like  VACP) 

double 

Vestibular  workload 

0-7 

OikeVACP) 

double 

Visual  workload 

0-7 

OikeVACP) 

double 

Olfactory  worMoad 

0-7 

OikeVACP) 

double 

Psychomotor  workload 

Oike  VACP)  1 

double 

Derived  Date 

Somatic  level 

0, 1.2,3 

double 

None,  low,  medium,  hi^ 

0, 1.2,3 

None,  low,  medium,  high 

V^tibular  level 

None,  low,  medium,  high 

Visual  level 

0. 1.2,3 

double 

None,  low,  medium,  high 

0.1. 2, 3 

double 

None,  low,  medium,  high 

0. 1.2,3 

double 

None,  low,  medium,  high 

Table  6.  External  world  state  parameters. 


World 

Parameter 

Date  Type 

Comments 

Raw  Date 

jjjjlfllllllBHHHH 

Time 

0  tOQO 

seconds 

Derived  Data 

Combat 

0  or  1 

binary 

Since  there  is  no  “horizon  visible”  sensor  on  any  aircraft  of  which  we  are  aware,  we  can  use 
OKCR  to  give  our  aiding  system  an  indication  of  pilot  state  and  orientation  with  the  visible  hori¬ 
zon.  If  the  aircraft  is  banked  and  the  pilot’s  head  position  indicates  OKCR,  then  we  can  assume 
that  the  horizon  is  visible.  A  lack  of  OKCR  may  indicate  that  there  is  no  visible  horizon.  In  any 
event,  pilot  head  position  is  important  to  our  modeling  and  coxmtermeasures  aiding  because  it  is 
part  of  the  assessment  of  pilot  susceptibility  to  SD  (especially  Coriolis). 

Micro  Saint  Model  -  Aircraft,  Pilot,  World 

For  Phase  I,  a  computer  model  is  used  in  place  of  a  simulator  or  aircraft  to  simulate  tiie  states  of 
the  aircraft  and  the  pilot.  The  model  functions  by  using  tiie  data  from  the  scenarios  and  supply¬ 
ing  tiiat  data  to  the  State  Table  in  much  the  same  way  that  data  would  be  supplied  by  a  fli^t 
simulator  or  actual  aircraft.  The  SD  Aiding  System  then  uses  the  data  firom  fee  State  Table  to 
determine  if  fee  pilot  is  experiencing  SD.  The  aircraft  and  pilot  models  are  in  fee  latest  version 
of  Micro  Analysis  &  Design’s  discrete  event  simulation  tool  called  Micro  Saint  Sharp. 

The  first  step  was  to  convert  fee  scenario  data  fi-om  fee  spreadsheet  format  shown  in  Chapter  V 
(Table  3)  to  Micro  Saint  Sharp  code.  Each  row  in  fee  spreadsheets  represente  a  data  event  feat 
details  fee  state  of  the  aircraft  and  pilot  for  a  specific  time  period.  Each  successive  data  event 
represents  state  changes  as  time  advances.  An  Excel  macro  was  developed  to  take  fee  data  for 
each  event  and  to  transpose  it  into  a  two-dimensional  data  array.  The  array  holds  fee  values  for 
each  state  variable  for  each  timed  event.  For  example,  fee  code  ‘‘ScenarioData[3,  5]  =  0.43868;'‘ 
is  the  data  fi-om  fee  third  timed  event  and  fee  fifth  column  in  fee  spreadsheet  corresponding  to 
fee  roll  angle.  The  code  representing  fee  full  data  set  for  one  of  the  60-second  scenarios  is  over 
9000  lines  long.  The  duration  of  each  event  is  calculated  based  on  fee  elapsed  time  fi-om  the  first 
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column  of  the  spreadsheet.  The  data  was  recorded  from  X-plane  at  about  8  Hz,  so  the  duration 
for  each  data  event  is  about  1/8  of  a  second. 

The  Micro  Saint  Sharp  model  is  a  combination  of  a  flight  scenario  data  in  the  ScenarioData 
array  and  a  task  network  diagram.  The  basic  network  diagram  (Figure  13)  is  a  combination  of 
object  nodes  and  connections.  The  nodes  represent  events  while  the  connections  represent  the 
basic  flow  through  the  network.  Each  node  is  associated  with  a  time  period  that  advances  the 
simulation  clock  and  that  contains  code  to  change  the  values  of  state  variables  within  the  model. 
The  first  node  is  simply  a  placeholder  to  begin  the  model.  The  second  node  is  used  to  update  the 
state  variables  from  the  scenario  for  each  time  period.  Two  different  paths  lead  from  tliis  node 
showing  that  the  network  flow  can  continue  back  to  itself  (the  small  looped  arrow  on  top  of  node 
2)  or  to  the  end  scenario  node.  Each  time  node  2  is  executed,  it  will  reference  the  next  data  event 
from  the  scenario  data  set,  and  send  that  data  row  to  the  State  Table.  This  is  analogous  to 
receiving  flight  data  from  a  flight  simulator  or  a  real  aircraft.  This  process  continues  until  all  the 
scenario  data  events  have  been  executed. 


.  . 

End  Sc&naric  V? 

Figure  13.  Micro  Saint  model. 


Node  2  also  advances  the  simulation  clock  by  the  amoimt  of  the  duration  of  the  data  event. 

Along  with  the  flight  and  pilot  data,  the  value  of  the  simulation  clock  is  sent  to  the  State  Table, 
which  functions  as  the  SD  Aiding  System’s  clock.  The  State  Assessor  uses  this  clock  to  deter¬ 
mine  the  timed  sequence  of  events  for  assessing  SD  (for  example.  Figure  9  in  Chapter  IV  ). 

For  the  Phase  I  demonstration,  the  simulation  was  used  in  a  deterministic  mode  in  which  the  time 
periods  and  state  values  for  each  event  did  not  vary  across  different  model  executions.  This 
allowed  us  to  repeatedly  simulate  exactly  the  same  disorientation  sequence  in  order  to  evaluate 
the  proper  execution  of  the  rest  of  the  SD  Aiding  System.  The  simulation  could  also  be  executed 
stochastically  such  that  the  time  and  state  values  would  vary  randomly  within  some  distribution. 
This  would  allow  the  model  to  represent  variations  to  the  original  scenarios  to  help  determine  the 
sensitivity  and  accuracy  of  our  SD  illusion  models. 

The  Micro  Saint  Sharp  model  also  includes  workload  score  calculations.  These  workload  scores 
are  stored  in  the  State  Table  and  used  by  the  Coxmtermeasure  Assessor  to  determine  the  potential 
effectiveness  of  different  compensatory  actions  in  a  specific  SD  situation.  We  derive  our  work¬ 
load  assessment  from  Wickens’  Multiple  Resource  Theory,  and  add  other  sensory  channels  to 
Wickens’  visual,  auditory,  and  psychomotor  set.  The  resulting  set,  which  we  call  SAWOY, 
represents  somatic,  auditory,  visual,  vestibular,  olfactory,  and  psychomotor  workload  scores 
For  psychomotor,  we  have  primary  and  secondary  observable  actions.  Primary  actions  are  the 
pilot’s  flight  control  actions.  Secondary  actions  are  everything  else  (such  as  pushing  buttons  or 
Turning  knobs).  We  also  infer  cognitive  cffuit  tu  eumplctc  the  model  of  pilot  workload. 
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Assessors 

Once  tiie  State  Table  is  filled,  the  SD  Aiding  System  uses  this  information  to  determine  the 
extent  to  which  the  pilot  is  experiencing  spatial  disorientation  and  the  best  way  to  help  the  pilot 
rwover.  The  assessore,  which  are  part  of  the  State  Table,  do  the  first  part  of  this  task.  They 
determine  the  extent  to  which  the  pilot  is  spatially  disoriented  and  the  risk  surrounding  the  cur¬ 
rent  situation.  These  different  assessors  within  the  State  Table  will  be  referred  to  as  the  State 
Assessor  in  the  rest  of  this  description  of  the  SD  Aiding  System  prototype.  The  Countermeasure 
Assessor  determines  the  appropriate  compensatory  actions,  based  on  the  outputs  of  the  State 
Assessor. 

As  mentioned  earlier,  the  SD  Aiding  System’s  architecture  relies  upon  a  newly  developed  soft¬ 
ware  product.  Micro  Saint  Sharp:  This  software  has  two  features  that  the  SD  Aiding  System 
uses.  The  first  feature  was  previously  introduced  in  the  State  Table  subsection;  it  is  a  base  plug¬ 
in  interface  that  is  used  for  transferring  data  fi'om  the  Micro  Saint  Sharp  model  to  other  modules 
of  the  SD  Aiding  System.  The  second  feature  is  a  Micro  Saint  Sharp  wrapper  class  that  allows 
for  file  rapid  prototyping  of  C#  software.  The  Micro  Saint  Sharp  wrapper  supports  having  all  the 
SD  Aiding  System  components  in  one  piece  of  software,  including  the  pilots aircraft  model 
(Figure  14). 


State  Assessor 

The  State  Assessor’s  primary  task  is  to  detect  an  SD  situation  by  monitoring  the  current  states  of 
the  pilot,  aircraft,  and  world  via  the  data  in  the  State  Table.  There  are  three  fimctions  that  work 
togettier  to  estimate  the  pilot’s  spatial  disorientation  and  the  severity  of  the  situation  (Figure  15): 
one  for  the  pilot,  one  for  the  aircraft,  and  one  for  the  world. 

SD  Assessor 

The  first  function  is  the  SD  Assessor,  which  monitors  the  pilot’s  vestibular  system  relative  to  the 
aircraft  attitude  using  the  SD  models  of  Leans  and  Coriolis.  Its  primary  goal  is  to  estimate  if  the 
pilot  has  become  spatially  disoriented,  what  type  of  illusion  he  or  she  may  be  experiencing,  and 
the  severity  of  the  illusion.  This  is  based  on  how  the  pilot’s  vestibular  system  reacts  to  the  air¬ 
craft's  aiiguliu  ttucclcrations  aiid  the  pilot's  head  movomento. 
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Figure  15.  State  Assessor. 


The  SD  Assessor  is  executed  each  time  the  State  Table  is  updated  with  new  data  about  the  state 
of  the  aircraft  or  the  pilot.  For  the  two  demonstration  scenarios  (Chapter  V),  this  can  be  as  fi-e- 
quently  as  8  times  per  second  of  simulation  time.  The  Leans  and  Coriolis  models  are  coded  as  a 
series  of  if-then  statements  and  flags  used  to  determine  if  the  progressing  state  of  the  aircraft  and 
pilot  match  the  SD  trigger  events  from  the  models.  As  described  previously,  we  use  the 
Mulder’s  constant  to  determine,  for  each  data  event,  if  the  maneuvers  of  the  aircraft  have 
exceeded  the  vestibular  threshold  for  pitch  and  roll  (we  ignore  yaw,  for  now).  Both  models  in 
the  SD  assessor  use  this  calculation  when  evaluating  the  state  of  the  vestibular  system. 

Leans  Assessor 

Since  the  Leans  model  is  based  on  a  sequence  of  SD  triggering  events,  the  code  must  keep  track 
of  which  events  have  occurred  within  the  sequence.  In  addition,  it  must  determine  if  trigger 
events  thftt  have  already  occurred  are  still  valid.  Each  time  the  State  Table  is  updated,  the  Leans 
Assessor  begins  by  looking  for  a  roll  rate  that  the  vestibular  system  does  not  notice  based  on  the 
rate  and  duration  being  below  Mulder’s  constant.  For  the  next  State  Table  update,  it  first  checks 
that  any  maneuvers  have  exceeded  Mulder’s  constant.  If  Mulder’s  constant  is  exceeded  (i.e.,  the 
motion  is  detected),  the  Leans  Assessor  resets  the  first  trigger  and  starts  over  again.  But,  if 
Mulder’s  is  still  not  exceeded,  the  Leans  Assessor  then  looks  for  a  roll  angle  of  greater  than  5 
degrees.  As  the  scenario  continues,  if  the  pilot’s  vestibular  system  is  still  not  reacting  based  on 
the  Mulder’s  calculation  and  the  roll  angle  stays  at  greater  than  5  degrees  for  greater  than  5  sec¬ 
onds,  the  assessor  will  determine  that  a  Leans  SD  has  begun  and  will  tell  the  rest  of  the  system 
that  a  Leans  is  occurring  with  a  low  level  of  confidence.  If  these  conditions  continue,  the  asses¬ 
sor  will  check  to  see  if  the  aircraft  is  losing  altitude.  If  it  is,  then  the  assessor  will  increase  the 
confidence  level  to  medium.  If  these  conditions  continue,  the  assessor  will  look  for  a  roll  rate 
above  the  threshold  of  the  vestibular  system  based  on  the  Mulder’s  calculation.  If  this  occurs, 
the  assessor  will  increase  the  SD  confidence  level  to  high.  Finally,  if  all  these  conditions  have 
occurred  in  sequence  and  the  pilot’s  head  position  deviates  in  the  opposite  direction  fi-om  the  roll 
back,  the  SD  confidence  level  is  increased  to  very  high. 

The  second  SD  trigger  event  requires  that  a  roll  of  greater  than  5  degrees  be  held  for  greater  than 
5  seconds.  During  the  evaluation  of  the  demonstration  scenario,  it  is  quite  common  for  the  5- 
degree  roll  event  to  begin  but  not  last  the  required  5  seconds  before  reducing  to  less  than  5 
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degrees.  When  this  occurs,  the  assessor  again  determines  if  the  first  event  was  still  valid  and  for 
the  initiation  of  the  second  event.  In  this  manner,  the  Leans  Assessor  keeps  constant  track  of  the 
state  of  the  aircraft  and  pilot  relative  to  the  Leans  model  event  sequence. 


Coriolis  Assessor 

The  coding  for  the  Coriolis  Assessor  is  simpler  than  the  Leans  because  the  Coriolis  Model 
requires  only  a  simultaneous  combination  of  two  events.  Each  time  the  State  Table  is  updated, 
the  assessor  checks  to  see  if  the  Mulder’s  constant  has  been  exceeded  in  any  of  the  three  planes 
(yaw,  pitch  or  roll)  and  if  the  pilot’s  head  is  turned  greater  than  30  degrees  in  either  of  the  other 
two  planes.  (In  the  current  prototype,  we  ignored  yaw  due  to  the  lack  of  yaw  data.)  If  these  two 
events  occur  simultaneously,  ftie  Coriolis  Assessor  determines  that  an  SD  may  be  occurring  and 
sets  the  confidence  level  at  medium.  For  example,  if  the  aircraft  is  changing  pitch  fast  enough  to 
exceed  the  Mulder’s  constant  for  pitch  and  the  pilot’s  head  is  turned  greater  than  30  degrees  in 
the  roll  plane,  the  assessor  would  determine  that  the  pilot  could  begin  to  encounter  the  effects  of 
the  Coriolis  Illusion.  Unlike  the  Leans  Assessor,  the  Coriolis  Assessor  does  not  keep  track  of 
any  previous  events  or  check  if  other  events  are  occurring. 

Table  7  summarizes  the  previous  two  paragraphs. 


Table  7.  State  Assessor  outputs 


Parameter 

Ranse 

Units 

Data  Type 

Comments 

Confidence  level 

0. 1,2,3 

None,  Low,  Moderate,  High 

Illusion  t^e 

0, 1,2 

None,  Leans,  Coriolis 

Trigger  event 

0, 1, 2, 3, 4, 5,  6, 7, 8, 

9 

None,  Sub-threshold  Roll  Rate,  Roll 
Greater  than  5  degrees.  Loss  of 
Altitude,  Sudden  Reverse  Correction, 
Reverse  Neck  Rotation,  Head  Roll  or 
Yaw,  Head  Pitch  or  Yaw,  Head  Roll 
or  Pitch  • 

When  an  SD  event  has  been  detected,  the  system  must  determine  the  severity  of  the  pilot’s  situa¬ 
tion  (Figure  16).  The  importance  of  SD  severity  can  be  seen  in  the  SD  remediation  network. 
This  network  implies  that  an  increasing  severity  of  SD  warrants  an  increasing  level  of  aiding. 
This  is  where  the  other  components  of  the  State  Assessor  become  important. 


Defining  conditions: 

-  Aircraft  state 

-  External  conditions 

-  Pilot  state 


Defining  conditions: 

-  Aircraft  stBte 

-  External  conditions 

-  Pilot  state 


Defining  conditions: 

-  Aircraft  state 

-  External  conditions 

-  Pilot  state 


SD  possible 


dkected 


Types  of  aiding 


Types  of  aiding 


Types  of  aiding 


Recovery 


Increasing  severity  of  SD 


Figure  16.  SD  remediation  network. 
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Aircraft  Assessor  &  World  Assessor 

The  Aircraft  and  World  Assessors  help  the  State  Assessor  estimate  the  severity  of  the  pilot’s 
situation  by  analyzing  aircraft  and  world  state  data.  SD  events  can  be  divided  into  three  primary 
levels  of  severity,  where  each  level  infers  the  goal  of  the  SD  Aiding  System.  These  levels  are: 

•  Onset  situations:  Emphasis  on  prevention  (Figure  17) 

•  Evolving  situations:  Emphasis  on  helping  pilot  recognize  and  recover  (Figure  1 8) 

•  Critical  situations:  Emphasis  on  recovery  (Figure  19) 


Types  of  Aiding; 

-  Mormal  displays  (HDD, 

.  HUD,  HMD,  low  tactile) 

-  No  audio,  olfactory  displays 


Types  of  Aiding; 

^  Enhanced  displays  (HDD, 
HUD,  HMD,  low  tactile) 

-  Low  audio 

-  No  olfactory  display 


Types  of  Aiding: 

:>  Enhanced  displays  (HDD, 
HUD,  HMD) 

-  Medium  tactile 
“  Low  audio 
“  No  olfactory  display 
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~  Pilott^Tid  problems  . 
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Defining  States: 

“  Aircraft:  controlled  flight 
“  External:  fair  visibility 
-  Pilot:  no  problems  . 


'  'i  /  ^ 

-^Horizon  'Gone  ^ 


Defining  States: 

-  Aircraft:  controlled  flight 

-  External:  poor  visibility 

“  Pi  lot :  poss  ibly  d  istracted 


Figure  17.  SD  Onset  remediation  network. 


The  State  Assessor  monitors  for  situations  that  fit  into  the  second  and  third  levels.  Evolving 
(Figure  18)  and  Critical  situations  (Figure  19). 


Figure  18.  SD  Evolving  remediation  network. 


45 


F33615-03-M-6360 


Multisensory  Integration  for  Pilot  Spatial  Orientation 


The  Aircraft  Assessor  monitors  the  aircraft’s  state.  Its  primary  goal  is  to  estimate  the  situation’s 
current  risk  based  on  the  aircraft’s  acceleration  and  position.  For  instance,  the  pilot  is  in  a  more 
severe  situation  if  the  aircraft  is  in  an  unusual  attitude  and  about  to  crash  (Figure  19),  and  this 
assessor  monitors  the  aircraft’s  time  until  impact  with  the  terrain.  The  less  time  the  pilot  has  to 
recognize  and  r^over  fi*om  SD,  the  more  intrusive  Ihe  countermeasures  must  be. 


Types  of  Aiding: 

-  SD  icons  on  displays 

-  High  tactile 

-  High  audio 

“  Olfactory  display 


Types  of  Aiding: 

-  SD  Icons  on  displays 

-  High  tactile  ^ 

-  Olfactory  display 

-  Audio  recovery  commands 

-  Auto-recovery^  If  commanded 


Types  of  Aiding: 

-  SD  Icons  on  displays 

-  High  tactile 

-  Olfactory  display 

-  Audio  recovery  commands 

-  Auto-recovery 

-  Auto-ejectlon,  If  authorized 


Defining  States: 

-  Aircraft:  unusual  attitude  with 
significant  alrepeed  loss  or  gain 

-  External:  no  combat  reason 

-  Pilot:  significant  stress 


<  '  '  VO 


Defining  States: 

-  Aircraft:  at  flight  envelope  limit 

-  External:  no  combat  reason 

-  Pilot:  "Help!"  "Recover!" 


Defining  States: 

-  Aircraft:  imminent  ground 
Impact  or  departure  from 
controlled  flight 

-  External:  no  combat  reason 

-  Pilot:  Incapacitated 


Figure  19.  SD  Critical  remediation  network. 


The  World  Assessor  monitors  the  external  world’s  state.  In  particular,  it  is  concerned  with  fac¬ 
tors  that  contribute  to  or  increase  the  pilot’s  vulnerability  to  SD.  These  factors  include  visibility 
(is  the  horizon  visible?),  and  the  presence  of  enemy  threats.  One  example  is  that  pilots  are  more 
susceptible  to  SD  if  they  cannot  see  the  real  horizon.  Another  example  is  that  we  may  suspend 
countermeasure  actions  if  the  pilot  is  jinking  to  avoid  an  enemy  missile. 

These  types  of  calculations  are  difficult  to  do,  because  the  required  parameters  are  hard  to  meas¬ 
ure  with  the  technology  that  is  currently  available  in  an  aircrauft.  We  do  not  perform  calculations 
in  the  prototype  that  we  are  not  able  to  do  with  data  provided  by  a  real  simulator  or  aircraft. 
Consequently,  the  analyses  that  occur  in  this  portion  of  State  Assessor  are  conservative.  These 
concepts  are  included  in  our  design  to  ensme  completeness. 

Countermeasure  Assessor 

The  Countermeasure  Assessor  uses  the  SD  type  and  level  firom  the  State  Assessor  to  determine 
what  type  of  cues  are  appropriate  countermeasures.  It  also  uses  worklo^  levels  (i.e,,  SAWOY 
scores)  firom  the  State  Table  to  determine  the  predicted  effectiveness  of  each  countermeasure. 

Once  the  Countermeasure  Assessor  has  selected  appropriate  coimtermeasures  for  the  SD  situa¬ 
tion,  it  sends  this  infonnation  to  the  (simulated)  aircraft  displays.  It  also  stores  which  counter¬ 
measures  it  is  using  in  the  State  Table,  so  that  tiie  State  Assessor  knows  which  pilot  resources  are 


46 


F33615-03-M-6360 


Multisensory  Integration  for  Pilot  Spatial  Orientation 


being  loaded.  The  following  table  contains  the  type  of  countermeasures  the  assessor  could  select 
to  help  the  pilot  recover  from  an  SD  event. 


Table  8.  Countermeasure  Assessor  outputs. 


Aid  Outputs 

Parameter 

Range 

Units 

Data  Type 

Comments 

Display  status 

normal,  enhanced 

SD  icon 

Oorl 

displayed  or  not 

low,  medium,  high 

volume 

0  or  1 

active  or  not 

Tactile 

low,  medium,  high 

tactor  frequency  and  amplitude 

0  or  1 

active  or  not 

Auto-recovery 

Oor  1 

active  or  not 

Auto-ejection 

Oorl 

active  or  not 

If  the  State  Assessor  detects  a  change  in  the  current  SD  event,  it  alerts  the  Countermeasure 
Assessor  of  this  change.  For  instance,  if  the  State  Assessor  estimates  that  the  pilot  has  fully 
recovered  from  the  SD  event,  it  passes  this  information  to  the  Countermeasure  Assessor.  Upon 
receiving  this  information,  the  Countermeasure  Assessor  terminates  the  various  countermeasures 
it  initiated.  The  State  Assessor  also  informs  the  Countermeasure  Assessor  when  the  SD  situation 
has  either  decreased  or  increased  in  severity,  so  that  the  countermeasures  can  be  changed 
accordingly. 

Countermeasures 

The  prototype’s  Coimtermeasures  represent  technologies  currently  available  in  a  cockpit,  such  as 
helmet-mounted  displays  (HMDs),  as  well  as  new  technologies,  such  as  tactile  vests.  The  Coun¬ 
termeasures  are  represented  in  a  simplified  cockpit  image  (Figure  20)  for  proof-of-concept  dem¬ 
onstration  purposes. 

SD  Aiding  System:  Ciass  Architecture 

SDApplication: 

This  is  a  class  that  inherits  from  the  Micro  Saint  Sharp  wrapper  form  class.  Micro  Saint 
Sharp  models  can  be  opened  and  executed  in  it.  This  class/form  functions  as  an  executive 
that  opens  and  executes  all  the  components  of  the  SD  Aiding  System  via  a  graphical  inter¬ 
face.  This  includes  opening  the  Micro  Saint  Sharp  model  of  the  pilot,  airplane,  and  world. 

StateStoragePlugin: 

This  is  a  class  that  inherits  from  IPlugin  interface.  This  is  used  by  the  Micro  Saint  Sharp 
model  to  set  members  of  the  StateTable  class. 

StateTable: 

This  class  provides  access  to  the  raw  and  derived  data  for  the  pilot,  aircraft,  and  world.  It  has 
public  instances  ofthe  following  classes:  PilotTable,  AircraftTable,  WorldTable,  and 
CountetTneasurelable.  It  is  accessible  by  the  Micro  Saint  Sharp  model  (via 
StateStoragePlugin)  and  the  CountermeasureAssessor  class.  The  methods  used  to  determine 
if  a  pilot  is  spatially  disorientated,  AssessSDLeans  and  AssessSDCoriolis,  are  part  of  this 
class.  When  an  assessor  determines  that  the  pilot’s  SD  state  has  changed  (i.e.,  the  pilot  has 
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become  spatially  disoriented,  or  SD  has  decreased  or  increased,  or  the  pilot  is  no  longer  spa¬ 
tially  disoriented),  it  triggers  an  event  that  is  monitored  by  the  CountermeasureAssessor 
class.  For  each  triggered  event,  the  CountermeasureAssessor  determines  the  appropriate 
countermeasures  to  apply  based  upon  the  logic  illustrated  in  Figures  17-19. 
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F^ure  iorOemonstration  screen™(^^  front  cxsckpit)  with  output  ‘buttons*  for  multisensory  channels 

(left  column)  and  assessor  outputs  (right  column). 
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PUotTable: 

This  class  contains  the  raw  data,  data  calculations,  and  derived  data  for  the  pilot,  plus 
pilot  state  Msessor  functions,  which  assess  ttie  pilot’s  workload  level  (SAWOY  scores) 
and  current  head  position. 

AircraftTable: 

This  class  contains  the  raw  data,  data  calculations,  and  derived  data  for  the  aircraft,  plus 
an  aircraft  state  assessor  function,  which  assesses  the  aircraft’s  state  (i.e.  erratic  control, 
unusual  attitude,  etc.). 
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WorldTable: 

This  class  contains  the  raw  data,  data  calculations,  and  derived  data  for  the  world,  plus  a 
world  state  assessor  function,  which  assesses  the  world’s  state  (i.e.  visibility  level,  com¬ 
bat,  etc.) 

CountermeasureAssessor: 

This  class  determines  the  best  countermeasures  to  apply  based  on  input  from  an  instance  of  - 
the  StateTable  class.  It  also  uses  pilot  workload  data  (S AWOY  scores)  from  StateTable  to 
determine  which  countermeasures  might  not  be  detected  due  to  overload  in  a  particular  chan¬ 
nel  or  channels.  For  example,  if  the  pilot  is  talking  on  the  radio,  then  audio  cues  might  be 
less  effective  than  other  sensory  cues  (e.g.,  visual,  tactile). 

CountermeasureDisplays: 

This  class  uses  the  outputs  of  the  CountermeasureAssessor  to  make  the  appropriate  changes 
to  the  SD  Aiding  System  prototype’s  displays  (the  column  of  square  buttons  in  Figure  20). 


Conclusion 

Our  SD  Aiding  System  builds  upon  prior  intelligent  cockpit  systems,  so  we  have  confidence  that 
the  design  is  robust  and  suitable  for  a  range  of  applications,  including,  obviously,  spatial  disori¬ 
entation.  Our  demonstration,  which  includes  the  two  scenarios  described  in  Chapter  V  plus  a 
scenario  based  upon  actual  data  obtained  from  the  Air  Force  Safety  Center  proves  the  feasibility 
of  our  model-based  concept  and  approach.  The  demonstration  has  been  cleared  for  public 
release  and  is  available  on  CD.  Interested  readers  should  contact  the  primary  author  for  a  copy 
of  the  demonstration,  which  executes  on  Windows  NT,  2000,  and  XP,  and  requires  no  other 
software  to  run. 
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VII.  Major  Phase  I  Accomplishments 

For  this  Phase  I SBIR,  MA&D  wjcomplished  the  following  major  objectives: 

•  We  performed  a  literature  search,  including  Russian  literature,  which  formed  the  founda¬ 
tion  of  our  quantitative,  model-based  approach.  An  annotated  bibliography  is  Chapter 
IX,  and  is  hopefiilly  useful  to  other  spatial  orientation  and  disorientation  researchers. 

•  We  designed  and  developed  models  for  the  Leans  and  Coriolis  illusions.  The  Leans 
model  worked  with  the  actual  F-16  data  we  received  from  the  Air  Force  Safety  Center 
(AFSC)  within  the  last  weeks  of  this  project’s  technical  work.  We  are  especially 
delighted  that  our  SD  Aiding  system  is  robust  enough  to  work  with  this  data  set  in  a  sen¬ 
sible,  realistic  manner.  (For  the  actual  SD  flight  data,  given  to  us  by  AFSC,  we  took  the 
raw  data  parameters  that  we  needed,  and  added  small  random  numbers  for  other  parame¬ 
ters  not  found  in  that  data  set.  Then,  we  ran  the  data  through  our  SD  Aiding  System 
looking  for  a  Leans-like  event.  We  did  not  try  our  Coriolis  model  on  the  actual  data 
because  we  did  not  have  pilot  head  position  data.  Even  without  head  position  data,  our 
Leans  model  worked  because  the  5*'’  step  (Figure  9),  a  head  tilt,  only  increases  our  confi¬ 
dence  level;  it  is  not  required  for  a  Leans  assessment.) 

•  We  enhanced  Wickens’  (1984)  Multiple  Resource  Theory  to  account  for  the  other  senses 
critical  to  this  effort.  We  called  the  resulting  “theory”  SAWOY  in  honor  of  Dr. 
Wickens’  location  in  Savoy,  Illinois  (at  the  University  of  Illinois’  Aviation  Research 
Laboratory),  The  newly  included  senses  are  somatic,  vestibular,  and  olfactory.  A  model 
of  pilot  workload,  using  hypothetical  (but  realistic)  SAWOY  scores,  influenced  the  SD 
countermeasures  applied  by  our  SD  Aiding  System. 

•  We  invented  an  SD  icon  (Figure  7)  to  be  an  intuitive  representation  of  the  aircraft  pitch 

and  roll.  How  intuitive  it  actually  is  remains  to  be  seen  via  testing  in  Phase  II. 

•  Most  importantly,  we  delivered  a  working  SD  Aiding  prototype  that  serves  as  an  excef- 
lent  foundation  for  Phase  II  development  and  testing  with  pilots  in  a  motion-based 
simulator. 
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VIII.  Phase  II  Directions 

Based  upon  our  success  in  Phase  I,  we  intend  to  pursue  two  major  thrusts  in  Phase  11: 

1 .  Enhance  and  validate  our  SD  illusion  models 

2.  Enhance  and  evaluate  our  SD  Aiding  System 
Each  of  these  thrusts  has  several  major  components. 

Model  Improvements 

To  enhance  om  SD  illusion  models,  we  will: 

•  Change  from  a  fixed  Mulder’s  constant  comparison  with  acceleration  duration,  to  a  more 
accurate  threshold  for  each  of  the  three  axes  (pitch,  roll,  and  yaw).  Those  thresholds  are 
3.2°/sec  for  roll,  2.6°/sec  for  pitch,  and  l.l°/sec  for  yaw.  Also,  utricle  sensing  of  linear 
accelerations  is  discussed  (Stapleford,  1968). 

•  Use  more  accurate  washout  values  for  pitch,  roll,  and  yaw  based  upon  the  latest  research 
by  Dr.  Brian  Self  and  colleagues  at  USAFA  (ongoing  experiments). 

•  More  thoroughly  test  our  models  with  actual  de-identified  flight  data  from  AFSC,  if  such 
data  sets  are  made  available  to  us.  If  we  receive  multiple  data  sets  from  AFSC,  we  will 
use  half  for  development  purposes,  and  the  other  half  for  testing.  Lessons  learned  from 
such  testing  will  be  used  to  make  our  models  more  robust. 

•  Vary  our  model  runs  stochastically  to  explore  boundary  conditions,  stability,  and 
accuracy  within  a  wider  range  of  test  conditions  than  can  be  done  with  non-varying 
X-Plane  or  AFSC  data  sets. 

•  Add  models  for  other  illusions  -  such  as  somatogravic.  Giant  Hand,  and  others  -  in 
consultation  with  our  customer.  The  decision  criteria  for  adding  illusions  will  be  based 
upon  the  illusions  frequency  of  occurrence  for  US  AF  (or  other  DoD)  pilots,  and  the 
illusion’s  dissimilarity  to  others  in  our  set,  so  as  to  cover  as  much  of  the  “illusion  spaee” 
as  practical.  If  actual  AFSC  data  suggests  an  illusion,  we  will  use  that  fact  as  another 
criterion. 

To  validate  our  models,  we  will  proceed  as  follows.  The  key  to  our  model  validation  is  the  crea¬ 
tion  of  a  disorienting  vestibular  illusion,  in  the  GAT-II  simulator,  as  represented  in  the  box  in  the 
left  center  of  Figure  21 .  Our  intention  for  initial  model  validation  is  to  create  the  Leans,  as  this  is 
both  common  and  a  precursor  to  more  serious  ineidents  of  SD  (e.g..  Graveyard  spin  and  spiral). 
Inputs  to  the  illusion  creation  box  are  three  factors: 

(1)  Scenario  parameters,  which  are  those  aspects  of  the  simulation  dynamics  that  are  effec¬ 
tive  in  creating  the  illusion.  These  include  three  aspects  of  bank  (primarily,  although  we 
might  consider  pitch  or  yaw):  (i)  the  degree  of  bank  that  is  achieved  following  straight  and 
level  flight,  (ii)  the  velocity  by  which  this  bank  is  achieved  (rate  of  change  of  bank),  and  (iii) 
the  time  duration  during  which  the  bank  is  changing  (duration  of  aircraft  rotation).  Note  that 
these  three  parameters  are  not  independent  of  each  other.  If  a  given  target  bank  angle  is  to  be 
achieved  following  an  X  second  duration  of  change,  this  will  dictate  the  rate  of  bank.  These 
three  parameters  can  be  combined  into  what  we  describe  as  a  “bank  profile”.  An  additional 
scenario  parameter,  wliieli  has  twu  states,  is  whether  the  bmik  ohemgc  is  aooomplishod 
actively  by  commands  given  to  the  pilot,  or  passively  by  the  simulation.  We  intend  to  vary 
these  scenario  parameters  across  a  series  of  SD  trials,  in  order  to  assess  which  combinations 
produce  which  levels  of  SD. 
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Figure  21.  Phase  II  model  validation  experimental  considerations. 


(2)  Modulating  factors  are  a  set  of  environmental  or  operating  conditions,  which  we  believe 
will  influence  the  onset  and  severity  of  SD.  These  include:  (i)  outside  world  (true  horizon) 
visibility,  which,  ideally,  will  be  varied  at  three  levels  -  perfect,  degraded  (haze),  and  absent. 
We  do  not  anticipate  that  perfect  visibility  will  produce  SD,  but  it  will  provide  us  with  a  set 
of  “control  condition”  measures  of  SD  =  0,  which  our  model  discriminates  from  the  SD  >  0 
measures;  (ii)  mental  workload  which  itself  will  be  discriminated  between  the  amount  of 
workload  (extraneous  task  demands)  and  the  source  of  workload  (visual  versus  non-visual 
distractions).  There  may  be  other  modulating  factors  we  wish  to  examine,  but  we  believe 
these  two  are  the  most  important 

(3)  Pilot  instructions,  such  as  “Maintain  heading  050  at  15,000  feet  and  300  knots.” 

Precisely  how  the  instructions  will  be  phrased  will  be  dictated  by  the  measurements  we  will 
take  (see  below). 

Assuming  that  we  can  design  a  scenario  that  can  both  produce  the  Leans  and  vary  the  magnitude 
of  its  experience,  via  the  parameters  described  above,  we  must  then  assess  the  pilot  state,  as 
shown  as  outputs  of  the  illusion  creation  box  in  Figure  21 .  These  assessments  are  reflected  both 
in  pilot  behavior  and  explicit  pilot  reports  of  SD.  PUot  behavior  includes  passive  measures  -  in 
particular  head  tilt,  which  will  give  us  a  measure  of  the  pilot’s  perceived  vertical  (as  inferred  by 
the  difference  between  the  perpendicular  to  the  perceived  horizon  in  IMC  and  the  true  horizon  in 
VMC).  In  later  simulations  another  important  passive  behavioral  measure  could  include  meas¬ 
ures  of  eye  scanning  (away  from  the  horizon).  It  is  possible  that  we  can  also  obtain  passive 
measures  of  head  rotetion,  to  infer  scanning  away  from  the  outside  world  and  the  attitude  indi¬ 
cator.  Pilot  behavior  also  includes  active  measures  of  flight  control  inputs  (i.e.,  stick  or  yoke 
deflection).  These  are  particularly  important,  as  they  will  assess  the  moment  of  active  correction 
from  an  undesirable  bank. 

Pilot  reports  are  the  key  assessments  of  the  SD  state  against  which  our  model  will  be  validated. 
Manual  reports  will  involve  periodic  assessments  of  the  pilot’s  belief  in  the  direction  of  vertical. 
These  can  be  provided  by  a  “sky  pointer”  held  in  the  left  hand,  which  can  be  rapidly  oriented 
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toward  the  pilot’s  belief  of  which  direction  is  up.  This  will  provide  an  objective,  quantifiable 
measure  of  DA.  The  orientation  can  be  videotaped  or  assessed  by  other  sensors.  Verbal  reports 
will  be  obtained  by  asking  the  pilot  to  periodically  rate  his  or  her  degree  of  certainty  in  spatial 
orientation  along  a  10-point  scale.  These  ratings  might  be  solicited  at  the  same  time  as  the  DA 
arrow  task.  They  will  provide  an  estimate  of  VAP. 

The  experimental  design  will  involve  manipulating  the  input  variables  on  the  left  of  Figure  21  to 
create  several  conditions.  It  remains  imclear  the  extent  to  which  a  single  pilot  will  experience  all 
conditions.  It  is  likely  that  some  of  the  experimental  variables  (such  as  ambient  workload)  will 
be  varied  between  pilots.  Also  unclear  is  the  amount  of  experience  that  we  will  need  firom  sub¬ 
ject  pilots.  These  uncertainties  will  be  defined  as  we  work  with  the  simulator  early  in  Phase  II  to 
develop  credible  scenarios  and  appropriate  ways  of  measuring  DA  and  VAP. 


SD  Aiding  System  Improvements 

To  enhance  our  SD  Aiding  System,  improvements  will  include  the  following. 

•  Enhance  and  calibrate  our  SAVVOY  scores  to  more  accurately  assess  pilot  workload  in 
all  of  the  sensory  channels  in  which  we  are  interested. 

•  We  have  heard  anecdotally  that  humans  habituate  to  audio  or  tactile  cues  within  10-20 
seconds.  We  will  search  the  literature  for  this  claim.  If  verified,  we  will  constrain  our 
intelligent  audio  and  tactile  cuing  to  accoimt  for  this  habituation. 

To  evaluate  our  SD  Aiding  System,  our  steps  will  include  the  following. 

•  Test  it  with  as  much  actual,  data  as  practical. 

•  Perform  stochastic  testing  to  check  boundary  conditions. 

•  Conduct  pilot-in-the-loop  (PITL)  trials  with  components  of  the  system,  as  well  as  with 
the  complete  system. 

-  To  conduct  PITL,  we  will  need  approval  fi'om  USAFA’s  IRB. 

-  To  test  our  multisensory  countermeasures,  we  will  first  test  them  in  isolation  (or  use 
the  literature  to  establish  a  sensory  countermeasure’s  efficacy),  and  then  test  them  in 
combination  with  others  in  order  to  achieve  an  intelligent  multisensory  approach  that 
appears  to  be  the  best  combination  for  a  given  situation. 


Conclusion 

Phase  n  will  give  us  an  opportunity  to  calibrate  and  validate  our  models  and  to  test  our  SD 
Aiding  System  under  realistic  simulated  scenarios  with  Air  Force  pilots. 
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Beringer,  D.B.,  &  Ball,  J.D.  (2000).  HUDs  versus  HDDs:  A  comparison  of  formats  for 
presentation  of  highway-in-the-sky  primary  flight  displays.  San  Antonio,  TX:  Spatial 
Disorientation  Symposium.  Foimd  at:  http://www.spatiald.wpafb.af.mil/Primary%20Flight 
%20Displays/beringer.pdf 

‘Tilots  expressed  preference  for  a  wide-angle  view  in  turns  but  narrower  field  of  view  for 
straight-and-level  cmise  (less  cluttered).”  An  argument  for  context-sensitive  display 
formats. 

Beringer,  D.B.,  Williges,  R.C.,  &  Roscoe,  S.N.  (1975).  The  transition  of  experienced  pilots  to 
a  frequency  separated  altitude  indicator.  Human  Factors,  1 7, 401-414. 

Berkley,  W.E.,  &  Martin,  E.L.  (2000).  Spatial  disorientation  in  night  vision  goggle 
operations.  San  Antonio,  TX:  Spatial  Disorientation  Symposium.  Found  at: 
http://www.spatiald.wpafb.afmil/Primary%20Flight%20Displays/berkley.pdf 

The  impact  of  NVG  operations  on  SD  susceptibility. 

Berthoz,  A.,  Israel,  I.,  Georges-Francois,  P.,  Grasso,  R.,  &  Tsuzukn,  T.  (1995).  Spatial 

memory  of  body  linear  displacement:  What  is  being  stored?  Science,  269, 95-98. 

Beyer,  M.  (1985).  Spatial  disorientation  in  the  F-15.  In  Proceedings  of  the  Aircraft  Attitude 
Awareness  Workshop.  Wright-Patterson  AFB,  OH:  Flight  Dynamics  Laboratory  (October  8- 
10),  pp  1-4-1  to  1-4-5. 

Lists  factors  not  considered  major  contributors  to  SD,  some  formation  cues,  HUD 

deficiencies,  VMC  complacency,  and  urges  vertifuge  training. 

Boff,  K.R.,  &  Lincoln,  J.E.  (1988).  Engineering  data  compendium:  Human  perception  and 
performance.  Volume  1.  Wri^t-Patteison  AFB,  OH:  Harry  G.  Armstrong  Aerospace  Medical 
Research  Laboratory. 

Section  3.2  contains  the  foramlas  for  the  vestibular  system  responses  to  momentary  and 
prolonged  stimuli.  In  particular,  Mulder’s  constant,  the  threshold  for  angular  acceleration 
duration  (i.e.,  2°/sec),  as  well  as  adaptation  (habituation  or  recalibration)  times. 

Borah,  J.,  Young,  L.R.,  &  Curry,  R.E.  (1988).  Optimal  estimator  model  for  human  spatial 
orientSition.  Annals  of  the  New  York  Academy  of  Sciences,  545,51-73. 

“A  model  is  presented  to  predict  human  dynamic  spatial  orientation  in  response  to  mul¬ 
tisensory  stimuli.  Motion  stimuli  are  first  processed  by  dynamic  models  of  the  visual, 
vestibular,  tactile,  and  proprioceptive  sensors.  Central  nervous  system  function  is  mod¬ 
eled  as  a  steady  state  Kalman  filter  that  optimally  blends  information  finm  the  various 
sensors  to  form  an  estimate  of  spatial  orientation.  Where  necessary,  nonlinear  elements 
preprocess  inputs  to  fixe  linear  central  estimator  in  order  to  reflect  more  accurately  some 
nonlinear  human  response  characteristics.  Computer  implementation  of  the  model  has 
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shown  agreement  with  several  important  qualitative  characteristics  of  human  spatial  ori¬ 
entation.” 

Box,  G.  (1979).  Robustness  in  the  strategy  of  scientific  model  building.  In  R.L.  Laimer  &  G.N. 
Wilkinson  (Eds.),  Robustness  in  Statistics.  St.  Louis,  MO:  Aeademic  Press,  202. 

Braithwaite,  M.G.,  Beal,  K.G.,  Alvarez,  E.A.,  Jones,  H.D.,  &  Estrada,  A.  (1998).  The 
optokinetic  cervico  reflex  during  simulated  helicopter  flight.  Aviation,  Space,  and 
Environmental  Medicine,  5P(12),  1166-1173. 

“The  optokinetic  cervico  reflex  (OKCR)  is  a  recently  hypothesized,  visually  driven  reflex 
that  serves  to  stabilize  the  image  of  the  external  horizon  on  the  retina  during  roll  maneu¬ 
vers  in  high-performance  aircrafl.  Although  reported  anecdotally,  head  tilt  during  heli¬ 
copter  flight  has  not  been  studied  formally.  Such  research  is  required  to  determine  the 
full  impact  and  significance  that  it  may  have  on  the  flying  performanee  of  a  rotary-wing 
aviator.  The  aim  of  this  study  was  to  investigate  the  relationship  between  horizon  posi¬ 
tion  and  the  perception  of  orientation  and,  thus,  to  generate  vital  information  to  assess 
whether  OKCR  plays  an  important  role  in  spatial  disorientation.  A  UH-60  flight  simula¬ 
tor  study,  with  20  volunteer  pilots  participating,  was  performed  to  examine  the  effects  of 
this  reflex  during  day  flight  and  during  flight  with  night  vision  goggles  (NVGs).  The 
results  confirm  that  the  OKCR  occurs  during  simulated  helicopter  flight,  both  with  and 
without  NVGs.  As  with  previous  studies,  head  roll  increased  during  flight  under  visual 
meteorological  conditions  in  relation  to  an  increasing  aircraft  roll  angle  up  to  a  maximum 
sustainable  level  and  then  remained  constant.  Head  roll  did  not  oecur  during  flight  under 
instrument  meteorological  conditions.  The  presence  of  the  OKCR  will  impact  rotary¬ 
wing  operations.  Various  aspects  are  discussed,  and  recommendations  are  made  for 
future  research.” 

Braithwaite,  M.  G.,  Durnford,  S.J.,  Crowley,  J.S.,  Rosado,  N.R.,  &  Albano,  J.P.  (1998). 
Spatial  Disorientation  in  U.S.  Army  Rotary-Wing  Operations.  Aviation,  Space,  and 
Environmental  Medicine,  69(11),  1031-1037. 

Braithwaite,  M.G.,  Durnford,  S.F.,  Groh,  S.L.,  Jones,  H.D.,  Higdon,  A.A.,  Estrada„  4.,  & 
Alvarez,  E.A.  (1998).  Flight  simulator  evaluation  of  a  novel  flight  instrument  display  to 
minimize  the  risks  of  spatial  disorientation.  Aviation,  Space,  and  Environmental  Medicine, 

69(S),  733-742. 

Braithwaite  M.G,  Hudgens  J.J.,  Estrada,  A.,  &  Alvarez,  E.  A.  (1998).  An  evaluation  of  the 
British  Army  spatial  disorientation  sortie  in  U.S.  Army  aviation.  Aviation,  Space,  and 
Environmental  Medicine,  69(8),  727-32. 

“Following  didactic  instruction,  most  aircrews  are  able  to  experience  some  of  the  disori¬ 
enting  illusions  and  limitations  of  the  orientation  senses  in  a  variety  of  ^ound-based 
deviees.  In  order  to  reinforce  instruction  in  SD  within  the  environment  in  which  they 
operate,  British  Army  Air  Corps  helicopter  pilots  also  receive  an  airborne  demonstration 
of  the  limitations  of  their  orientation  senses  prior  to  rotary-wing  instrument  fli^t  train- 
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ing.  The  objective  of  this  assessment  was  to  determine  whether  the  SD  demonstration 
sortie  would  be  an  effective  adjunct  in  training  aircrew  in  SD  in  the  U.S.  Army.  There 
were  45  aviators  and  training  personnel  who  experienced  the  sortie  and  gave  their  opin¬ 
ion  in  questionnaires.  The  following  conclusions  were  made:  the  maneuvers  performed 
in  the  SD  demonstration  sortie,  and  the  sortie  overall,  were  extremely  effective  at  demon¬ 
strating  the  limitations  of  die  orientation  senses;  the  SD  sortie  attracted  a  significantly 
higher  rating  in  its  effectiveness  to  train  aviators  in  SD  than  all  the  currently  available 
methods;  the  introduction  of  the  sortie  into  the  initial  flight  training  syllabus  would  be  a 
distinct  enhancement  to  the  SD  training  of  aviators  and  associated  personnel;  and  the 
introduction  of  the  sortie  into  refiresher  training  in  field  units  would  also  be  an  advantage. 
Other  services  are  encouraged  to  consider  this  enhancement  to  the  SD  training  of 
aviators.” 

Brandt,  T.  (1997).  Cortical  matching  of  visual  and  vestibular  3-D  coordinate  maps.  Annals  of 

Neurology,  42, 983-984. 

Broomhall,  L.  (2003).  Spatial  disorientation:  Something  to  avoid.  MS&T Magazine,  3, 29-32. 

A  training  technology  review  article  that  describes  two  simulators  from  two  different 
manufacturers. 

Brown,  D.L.,  DeVilbiss,  C.A.,  Ercoline,  W.R.,  &  Yauch,  D.W.  (2000)  Post  roll  effects  on 
attitude  perception:  “Gillingham  Illusion.”  Aviation,  Space,  and  Environmental  Medicine,  71, 
489-495. 

Bryan,  L.A.,  Stonecipher,  J.W.,  &  Aron,  K.  (1954).  180-degree  turn  experiment  (Aeronautics 
Bulletin  #11).  Urbana,  IL:  University  of  Illinois,  Institute  of  Aviation. 

Describes  a  procedure  for  non-instrument  pilots  to  fly  out  of  IMC,  if  they  accidentally  fly 

into  it. 

Carretta,  T.R,  Perry,  D.C.,  Jr.,  &  Ree,  M.J.  (1996).  Prediction  of  situational  awareness  in  F- 

15  pilots.  International  Journal  of  Aviation  Psychology,  d(l),  21-41. 

“Situational  awareness  (SA)  is  a  skill  often  deemed  essential  to  pilot  performance  in  bofti 
combat  and  non-combat  flying.  A  study  was  conducted  to  determine  if  SA  in  U.S.  Air 
Force  F-15  pilots  could  be  predicted.  The  participants  were  171  active  duty  F- 15  A/C 
pilots  who  completed  a  test  battery  representative  of  various  psychological  constructs 
proposed  or  demonstrated  to  be  v^id  for  the  prediction  of  performance  in  a  wide  variety 
of  military  and  civilian  jobs.  These  predictore  encompassed  measures  of  cognitive  abil¬ 
ity,  psychomotor  ability,  and  personality.  Supervisor  and  peer  ratings  of  S  A  were  col¬ 
lected.  Supervisors  and  peers  showed  substantial  agreement  on  the  S  A  ratings  of  the 
pilots.  The  first  unrotated  principle  component  extracted  fi-om  the  supervisor  and  peer 
ratings  accounted  for  92.5%  of  the  variability  of  ratings.  The  unrotated  first  principle 
component  served  as  the  SA  criterion.  Flying  experience  me^ured  in  number  of  F-15 
hours  was  the  best  predictor  of  SA.  After  controlling  for  the  effects  of  F-1 5  flying  hours. 


58 


F33615-03-M-6360 


Multisensory  Integration  for  Pilot  Spatial  Orientation 


the  measures  of  general  cognitive  ability  based  on  working  memory,  spatial  reasoning, 
and  divided  attention  were  found  to  be  predictive  of  SA.  Psychomotor  and  personality 
measures  were  not  predictive.  With  additional  F-15  flying  hours  it  is  expected  that  pilots 
would  improve  their  ratings  of  SA.” 

Chelette,  T.L.  (2001).  Measuring  the  head  tilt  illusion  during  sustained  acceleration.  Human 
Systems  lAC  Gateway,  /2(3),  4-6. 

“Magnitude  of  roll  illusion  (in  degrees)  Roll  illusion  =  0.3397  x  arcsin{(G0.25  -  1)  x  sin 
[head  pitch]  x  sin  [head  yaw]}  Pitch  illusion  =  0.1491  x  arcsin{(G0.25  -  1)  x  sin[head 
pitch]  X  cos[head  yaw]}”  “Specifically,  an  upward  head  pitch  combined  with  a  head  yaw 
into  a  turn,  as  is  common  in  formation  flying,  can  result  in  a  sensation  of  underbank  and  a 
pitch-up  of  the  aircraft.  Intended  corrective  action  actually  overbanks  the  aircraft  with  a 
pitch  down,  causing  loss  of  altitude.  Downward  head  pitches  during  turning,  as  is  com¬ 
mon  during  bombing  or  strafing  runs,  can  cause  a  sensation  of  overbanking.  Intended 
corrective  action  actually  underbanks  the  aircraft,  causing  altitude  gain  that  could  lead  to 
midair  collision  when  in  formation  flight.” 

Cheung,  R.  (2000).  Non-visual  spatial  orientation  mechanisms.  San  Antonio,  TX:  Spatial 
Disorientation  Symposium.  Found  at:  http://www.spatiald.wpafb.af  mil/MechanismsStudies 
/cheung.pdf 

Post-rotational  decay  in  Roll  (4s)  <  Pitch  (7s)  <  Yaw  (16s),  measured  fi'om  duration  of 
post-rotational  decay  of  the  resulting  illusory  sensation  of  rotation.  There  is  a  greater  rate 
of  error  development  in  roll  than  in  pitch  and  yaw. 

Semicircular  canal  inadequacies:  Limited  threshold  of  vestibular  perception  0.14, 0.5  and 
0.5°/s^  for  yaw,  roll  and  pitch  respectively.  Perceived  angular  velocity  is  less  than  the 
actual  angular  velocity.  Absence  of  sensation  of  rotation  during  constant  velocity  rota¬ 
tion.  Apparent  sensation  of  rotation  in  the  opposite  direction  during  deceleration.  Per¬ 
sistent  apparent  sensation  of  rotation  in  the  opposite  direction,  after  physical  rotation  has 
actually  stopped.  Greater  rate  of  error  development  in  roll  >  pitch  >  yaw. 

Cheung,  R.,  &  Hofer,  K.  (2003).  Eye  tracking,  point  of  gaze,  and  performance  degradation 
during  disorientation.  Aviation,  Space,  and  Environmental  Medicine,  74(1),  11-20. 

“The  cognitive  cockpit  concept  has  been  proposed  as  a  potential  disorientation  counter¬ 
measure.  It  involves  monitoring  the  pilot’s  physiological,  behavioral  and  subjective 
responses  during  disorientation.  This  data  is  combined  to  provide  a  real  time  model  of 
pilot  state,  which  is  used  as  a  basis  for  optimizing  pilot  performance.  This  study  attempts 
to  investigate  whether  there  are  consistent  behavioral  or  physiological  markers  that  can 
be  monitored  during  a  specific  disorientation  scenario.  An  Integrated  Physiological 
Trainer  with  interactive  aircraft  controls  and  an  eye-tracking  device  was  employed. 
Fourteen  subjects  proficient  in  maintaining  straight-and-level  flight  and  who  have 
acquired  the  skills  in  changing  attitude  participated  in  the  study.  They  were  exposed  to  a 
fli^t  profile  consisting  of  straight-and-level  flying  and  change  in  attitude  without  expo- 
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sure  to  a  head  roll  (control  condition)  and  a  profile  with  exposure  to  a  head  roll  (experi¬ 
mental  conditions)  during  constant  yaw  rotation.  Flight  performance  parameters  and 
subjects'  eye  movements  and  point  of  gaze  behavior  were  monitored  continuously. 
Immediately  on  the  return  to  upright  head  position,  all  subjects  reported  a  strong  apparent 
pitch  displacement  that  lasted  <  or  =  20  s  and  a  lesser  sensation  of  lateral  movement. 
Significant  differences  (p  <  0.01)  were  noted  on  a  number  of  scanning  behaviore  between 
the  control  and  the  experimental  conditions.  The  appearance  of  Nystagmus  was  apparent 
as  indicated  by  the  number  of  involuntary  saccades  during  disorientation.  Flight  per¬ 
formance  decrement  in  the  experimental  conditions  was  reflected  by  a  significant  devia¬ 
tion  in  maintaining  airspeed  (p  <  0.01).  It  appears  that  the  pitch  illusion  consistently 
affects  visual  scanning  behavior  and  is  responsible  for  the  decrement  in  flight  perform¬ 
ance  observed  in  the  simulator.” 

Cheung,  R.,  Money  K.,  Wright,  H.,  &  Bateman,  W.  (1995).  Spatial  disorientation-implicated 

accidents  in  Canadian  forces,  1982-92.  Aviation,  Space,  and  Environmental  Medicine,  66(6), 

579-85. 

“In  a  recent  survey  of  CF 1 8  aircrew  human  factors,  44%  of  pilots  reported  experience 
with  spatial  disorientation  (SD),  of  whom  10%  had  experienced  more  than  3  episodes.  In 
order  to  investigate  further,  we  have  completed  a  retrospective  study  of  SD-implicated 
category  A  accidents  (where  an  aircraft  is  destroyed,  declared  missing,  or  damaged 
beyond  economic  repair)  in  the  Canadian  Forces  (CF)  during  1982-92.  An  overview  Of 
all  SD  occurrences  (including  accidents  and  incidents)  across  aircraft  types  is  also  pre¬ 
sented.  Information  was  gathered  concerning  the  genesis  and  severity  of  disorientation 
so  that  research  effort  and  pilot  training  could  be  appropriately  implemented.  Mishap 
investigation  summaries  involving  category  A  accidents  where  SD  was  implicated  were 
obtained  from  the  CF  Directorate  ofFlight  Safety  and  reviewed.  We  also  examined  in 
detail  the  Board  of  Inquiry  Reports  of  these  accidents.  The  role  of  disorientation  in  these 
accidents  was  assessed.  There  were  62  category  A  accidents  between  1982-92  and,  in  14, 
SD  had  been  assigned  as  a  possible  cause  factor  in  the  accident  records.  When  divided 
into  the  categories  of  Recognized  SD  (RSD),  Unrecognized  SD  (USD),  and  Incapacitat¬ 
ing  SD  (ISD),  all  but  two  fell  into  the  category  of  USD  (the  pilots  were  unaware  of  the 
disorientation).  Of  the  SD  accidents,  1 1  involved  a  total  loss  of  24  lives.  The  majority  of 
the  accidents  happened  during  the  day,  and  pilots'  cumulative  flying  experience  did  not 
appear  to  be  a  significant  factor.  According  to  our  assessment,  there  were  two  episodes 
of  vestibular  origin,  involving  the  somatogravic  illusion.  Thi«e  episodes  of  disorientation 
occurred  over  frozen  lakes,  one  over  glassy  water,  and  one  over  ocean.” 

Cohen,  D.,  Otakeno,  S.,  Previc,  F.H.,  &  Ercoline,  W.R.  (2001).  Effect  of  “inside-out”  and 

“outside-in”  attitude  displays  on  off-axis  tracking  in  pilots  and  nonpilots.  Aviation,  Space,  and 

Environmental  Medicine,  72(3),  170-176. 

“Pilots  employing  helmet-mounted  displays  spend  sustained  periods  of  time  looking  off- 
axis,  necessitating  the  inclusion  of  attitude  symbology  on  the  helmet  to  mamtain  spatial 
awareness.  We  examined  how  fimdamentally  different  attitude  references,  a  moving- 
horizon  (‘inside-out’)  or  a  moving-aircraft  (‘outside-in’),  affected  pilot  and  nonpilot  atti- 
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tude  control  when  looking  on-  or  off-axis.  Both  a  rear-view  and  a  side-view  outside-in 
perspective  were  depicted  to  investigate  the  effect  of  control-display  compatibility. ... 
Subjects  performed  a  compensatory  pitch-roll  tracking  task  either  looking  on-axis  or  90 
degrees  off-axis  using  three  symbologies:  1)  a  compressed  pitch  ladder  with  horizon 
line;  2)  a  3-D  aircraft  representation  viewed  from  the  rear;  and  3)  a  3-D  aircraft  repre¬ 
sentation  viewed  from  the  side.  Tracking  error  in  roll  and  pitch,  control  bias,  and  subjec¬ 
tive  ratings  were  collected  and  analyzed. . ..  There  was  no  significant  difference  in  the 
tracking  performance  of  U.S.  Air  Force  pilots  in  pitch  and  roll  using  the  inside-out  or 
outside-in  rear-view  formats  on-  and  off-axis,  although  they  preferred  the  inside-out  for¬ 
mat.  Nonpilots  tracked  significantly  better  using  the  outside-in  rear-view  format,  which 
they  also  preferred.  Both  groups  tracked  poorly  using  the  outside-in  side-view  format 
and  control-display  compatibility  had  no  important  effect.  Pilots  are  equally  adept  using 
outside-in  and  inside-out  displays.  Given  that  an  outside-in  display  may  better  reflect  a 
person’s  inherent  frame  of  reference  for  orientation  (as  evidenced  by  the  nonpilots’  supe¬ 
rior  performance  with  it),  the  results  seem  to  indicate  that  pilots,  through  experience, 
have  adapted  to  an  inside-out  frame  of  reference.” 

Collins,  D.L.,  &  Harrison,  G.  (1995).  Spatial  disorientation  episodes  among  F-15C  pilots 

during  Operation  Desert  Storm.  Journal  of  Vestibular  Research,  5(6),  405-10. 

“Spatial  Orientation  (SO)  under  flight  conditions  is  the  accurate  ‘integration’  of  sensory 
inputs  from  the  dynamic  aviation  enAdronment  that  result  in  safe  and  effective  goal- 
oriented  performance.  Insidious  sensory  mismatches  routinely  occur  during  flight, 
impeding  pilot  performance.  When  this  sensory  dissonance  occurs,  if  not  appropnately 
resolved,  it  will  result  in  perceived  or  actual  errors  in  aircraft  control  that  are  estimated  to 
cost  the  Air  Force  between  $150  and  $200M  per  annum  in  aircraft  accidents.  A  scientific 
survey  was  created  and  administered  to  96  F-15C  combat  pilots  after  their  return  from 
Desert  Storm.  The  survey  sought  to  determine  where  in  the  flight  profile,  and  under  what 
conditions,  spatial  disorientation  (SD)  episodes  occurred.  The  survey  consisted  of  multi¬ 
ple  choice  and  open-ended  questions.  The  analyses  of  the  data  revealed  that  visual  tran¬ 
sitions  from  inside  to  outside  the  cockpit  (or  the  reciprocal)  [e.g.,  target  acquisition] 
under  different  conditions  of  flight  were  associated  with  the  occurrence  of  SD  episodes. 
The  frequency  of  SD  episodes  varied  depending  on  visual  transitions  (or  no  visual  transi¬ 
tions)  and  types  of  flight  conditions  (for  example,  night-time  and  bad  weather).  This  SD 
survey  provided  flight  information  that  allowed  us  to  direct  research  to  those  areas  that 
were  problematic  during  combat  operations.” 

Cone,  S.M.,  &  Hassoun,  5.A.  (1992).  Attitude  awareness  enhancements  for  the  F-16  head-up 

display  (Final  report  ASC-TR-92-5017).  Wright-Patterson  AFB,  OH:  Aeronautical  Systems 

Center. 


The  F-16  System  Program  Office  identified  a  set  of  HUD  enhancements  for  evaluation; 
“(1)  extended  horizon,  (2)  ghost  horizon,  (3)  articulated  nose  down  pitch  bars,  (4) 
removal  of  the  2:1  pitch  scale  compression,  (5)  moving  nose  down  pitch  bar  tic  marks  to 
the  inside  of  the  pitch  bars,  (6)  modified  bank  angle  indicator,  and  (7)  modified 
zenith/nadir  symbols.”  Experimental  results  for  15  pilots  flying  simulated  unusual  atti- 
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tude  recoveries  showed  faster  reaction  times  and  strong  preference  for  modifications  1-5 
&  7.  Results  for  the  modified  bank  angle  indicator  were  mixed, 

Costello,  R.G.  (1976).  Continuous  compensatory  audio  manual  tracking.  In  Proceedings  of 
Twelfth  Annual  Conference  on  Manual  Control  (NASA  TM  X-73,170).  Moffett  Field,  CA: 
Ames  Research  Center,  406-421. 

Glider  pilots  may  use  an  audio  variometer  for  indicating  their  sink  or  climb  rate  so  that 
they  can  maintain  visual  scans  and  not  have  to  look  inside  at  their  vertical  speed  indica¬ 
tor.  Increasing  tone  pitch  indicates  climbing;  descent  is  noted  by  the  decreasing  pitch  of 
the  tone.  The  author  speculates  that  pilots  use  the  tone  for  coarse  assessments  of  their 
flight  strategy,  while  using  the  horizon  for  fine  tuning  their  path. 

Craig,  G.,  Brown,  A.,  Jennings,  S.,  &  Cheung,  R.  (2003).  Flight  test  of  three  attitude 
indicators  for  unusual  attitude  recovery.  Presented  at  Aerospace  Medical  Association's  Annual 
Meeting,  May  4-9, 2003,  San  Antonio,  TX. 

Fhght  test  compared  a  conventional  AI,  an  arc-segmented  attitude  reference,  and  a  novel 
asymmetric  AI.  During  unusual  attitude  recoveries,  the  asymmetric  AI  supported  quicker 
recoveries  and  fewer  errore. 

Davenport,  C.  (2000).  Spatial  disorientation:  The  USAF  experience,  FYl 99 1-FY2000:  How 
much  are  we  willing  to payl  San  Antonio,  TX:  Spatial  Disorientation  Symposium.  Found  at: 
http://www.spatiald,  wpafb.af.mil/Mishap%20Studies/Davenport.pdf 

The  top  10  contributing  factors  to  SD  mishaps  are:  attention  management,  judg¬ 
ment/decision-making,  mission  demands,  psychological  factors,  mental  fatigue,  visibility, 
training,  behavior,  personal/community  factors,  and  instrumentation. 

Davenport,  C.  (2003).  Personal  communication  (email  message  of  7/9/2003). 

Vestibular  threshold  of  greater  than  2  degrees  per  second  per  second  acceleration  nor¬ 
mally  results  in  a  sensed  motion.  However,  if  the  pilot  is  task  overloaded,  even  greater 
accelerations  may  not  be  perceived. 

DeHart,  R.L.,  &  Davis,  J.R.  (eds.)  (2002).  Fundamentals  of  aerospace  medicine  (3'^'*  ed.,  ISBN 
0-7817-2898-3).  New  York:  Lippincott  Williams  &  Wilkins. 

A  must  have  reference  for  any  serious  SO/SD  researcher.  Chapter  8,  Spatial  Orientation, 
by  A.  J.  Parmet  and  Kent  K.  Gillingham,  pages  1 84-244,  covers  all  of  the  essentials  for 
this  topic,  including:  translational  and  rotational  motion,  actions  and  reactions,  visual 
orientation,  vestibular  fimction,  perceptions,  other  senses  of  motion  and  position,  illu¬ 
sions,  the  dynamics  of  spatial  orientation  and  disorientation,  other  sensory  phenomena, 
situational  awareness,  and  motion  sickness. 
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Important  excerpts  are:  “the  actual  vestibular  perceptual  thresholds  are  ‘constant  except 
when  they  vary’”  (page  201),  which  supports  our  model-based  Monte  Carlo  approach  to 
modeling  the  vestibular  system.  “The  leans  [is]  the  most  common  of  all  vestibular  illu¬ 
sions  in  flight”  (page  221),  which  supports  our  first  scenario  focusing  on  the  Leans. 

Dieterich,  M.,  Bucher,  S.F.,  Seelos,  K.C.,  &  Brandt,  T.  (1998).  Horizontal  or  vertical 
optokinetic  stimulation  activates  visual  motion-sensitive  ocular  motor  and  vestibular  cortex  areas 
with  right  hemispheric  dominance:  An  fMRI  study.  5ram,  1479-1495. 

Duffy,  C.J.,  &  Wurtz,  R.H.  (1995).  Responses  of  monkey  MST  neurons  to  optic  flow  stimuli 
with  shifted  centers.  Journal  of  Neuroscience,  75(7),  5192-5208. 

Endsley,  M.R,  &  Rosiles,  S.A.  (1995).  Auditory  localization  for  spatial  orientation.  Journal  of 
Vestibular  Research,  5(6),  473-85. 

‘The  use  of  3D  auditory  technology  that  provides  localization  of  auditory  cues  presented 
through  headphones  is  proposed  as  a  means  of  providing  supplemental  information  to 
pilots  on  the  spatial  orientation  of  an  aircraft.  This  technique  shows  promise  for  reducing 
accidents  due  to  spatial  disorientation  associated  with  high  visual  load.  A  study  was  con¬ 
ducted  using  USAF  pilots  as  subjects  to  determine  desirable  inverted  question  mark  cue 
characteristics  for  accurately  localizing  auditory  cues  using  this  technique.  The  study 
examined  the  use  of  9  different  cue  types  at  each  of  2  frequency  levels.  It  was  found  that 
the  accuracy  of  subjects’  localization  of  cues  in  elevation  was  greatly  enhanced  by  the 
use  of  multidimensional  cues  that  provided  redundant  elevation  information  through 
varying  frequencies  and  distance  from  the  horizon  cues  in  addition  to  the  inherent  spatial 
location  information.  Trade-offs  in  azimuth  localization  accuracy  and  response  time 
were  noted,  however.  Recommendations  are  made  for  further  developing  and  investi¬ 
gating  this  technique  for  the  aircraft  spatial  orientation  task.” 

Engineering  Acoustics,  Inc.  (2003).  See:  http://www.navysbir.brtrc.com/cap/briefingsadmin/ 
ea.asp. 


Describes  the  tactile  situation  awareness  system  (TSAS)  and  claims  reduced  perceived 
workload  and  improved  “performance  in  spatial  tasks,  especially  in  the  presence  of  dis¬ 
tracting  influences.” 

Ercoline,  W.R.  (1997).  Introduction  to  spatial  disorientation  training.  Found  at:  www.sam. 
brooks.afmil/web/ram/SPATIALD/instrume.html. 

Ercoline,  W.R.  (1998).  fri  Minutes  of  the  2T^  meeting  of  the  DoD  flight  symbology  working 
group.  Moffett  Field,  CA:  NASA  Ames  Research  Center. 

Ercoline  asserts,  “Attitude  symbology  is  not  needed  on  HMDs  to  maintain  attitude 
awareness  when  looking  off-axis  to  find  and  track  a  target  during  restricted  visibility.” 
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Ercoline,  W.R.  (2000).  The  attitude  component  of  the  primary  flight  reference:  Where  do  we 
go  from  here?  San  Antonio,  TX;  Spatial  Disorientation  Symposium.  Found  at:  http://www. 
spatiald.wpafb.af.miI/Primary%20Flight%20Displays/ercolineb.pdf. 

Ercoline  reports  a  study  of  3  different  attitude  displays  for  use  with  an  HMD.  The  so- 
called  grapefruit  presentation  had  die  best  results  in  terms  of  reaction  time,  error  rates  and 
subject  preference.  Furthermore,  the  Class  A  mishap  rate  due  to  SD  is  largely  unchanged 
since  the  1970s. 

Ercoline,  W.R.,  DeVilbiss,  C,A.,  &  Lyons,  T.J.  (1994a).  Trends  in  USAF  spatial 
disorientation  accidents  -  1958-1992.  In  Proceedings  of  the  SPIE  Conference  (A95-12001-01- 
54  in  Orlando,  FL).  Bellingham,  WA:  Society  ofPhoto-Optical  Instrumentation  Engineers, 
257-260. 

The  rate  of  major  aircraft  accidents  from  the  period  1958-1971  w^  5.36  per  100,000 
flying  hours;  the  rate  was  2.22  during  the  period  1 972-1992.  Even  thou^  the  accident 
rate  decreased,  the  rate  of  SD-related  accidents  remained  about  the  same:  0.32  and  0.35. 

Ercoline,  W.R.,  Freeman,  J.E.,  Gillingham,  K.K.,  &  Lyons,  T.J.  (1994b).  Classification 
problems  of  US  Air  Force  Spatial  Disorientation  accidents,  1989-91.  Aviation,  Space  and 
Environmental  Medicine,  65,  147-152. 

Ercoline,  W.R.,  &  Previc,  F.  (2001).  Spatial  disorientation,  geographic  disorientation,  loss  of 

situation  awareness,  and  controlled  flight  into  terrain.  Human  Systems  I  AC  Gateway,  72(3),  10- 

12. 

Spatial  disorientation  related  to  the  loss  of  situation  awareness. 

Ercoline,  W.R.,  Self,  B.P.,  &  Matthews,  R.S.  (2002),  Effects  of  three  helmet-mounted 
display  symbologies  on  unusual  attitude  recognition  and  recovery,  Aviation  Space  and 
Environmental  Medicine,  Nov;73(ll):  1053-8. 

“Helmet-mounted  displays  (HMDs)  allow  pilots  to  view  aircraft  instrument  information 
while  looking  to  the  side,  away  from  the  aircraft  centerline  axis.  In  that  situation,  pilots 
may  lose  attitude  awareness  and  thus  develop  spatial  disorientation.  A  secondary  con¬ 
cern  is  flie  possible  effects  of  visual  conflict  between  tiie  apparent  motion  of  traditional, 
nose-referenced  flight  symbology  and  the  off-axis  view  of  the  outside  world.  Alternative 
symbologies  will  provide  improved  attitude  awareness  for  HMDs  when  compared  with 
the  conventional  inside-out  symbology  now  used  with  head-up  displays  (HUDs),  if  the 
HUD  symbology  is  used  on  a  HMD.  The  9  pilots  were  presented  48  randomly  arranged 
unusual  attitude  conditions  on  a  HMD.  The  three  symbologies  included:  1)  the  inside- 
out  representation  now  used  with  fixed  HUDs,  which  features  a  moving  horizon  and  pitch 
ladder;  2)  an  outside-in  display  that  depicts  a  moving  aircraft  relative  to  a  fixed  horizon; 
and  3)  an  inside-out  novel  symbology  termed  the  grapefinit'  display  (OD).  The  back¬ 
ground  scene  contained  a  mix  of  either  a  front  view  orientation  or  a  side  view  one.  Con¬ 
ditions  were  randomized  within  and  across  subjects.  Subjective  preferences  were  col- 
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lected  after  the  completion  of  all  tasks.  Analysis  of  variance  repeated  measures  design 
revealed  that  stick  input  for  the  GD  was  significantly  faster  with  fewer  roll  reversal  errors 
than  either  of  the  other  two.  The  time  to  recover  to  straight  and  level  was  significantly 
shorter  for  the  fi'ont  view  orientation  than  the  side  view.  Of  the  nine  pilots,  eight  pre¬ 
ferred  the  GD  symbology  as  a  method  of  presenting  attitude  information  on  the  HMD. 
Results  suggest  the  current  HUD  symbology  is  not  the  best  way  of  displaying  attitude 
information  on  the  HMD.  Given  the  conditions  of  this  study,  the  best  way  of  presenting 
the  pilot  with  attitude  information  on  the  HMD  is  with  the  GD  symbology.” 

Eriksson,  L.,  Johansson,  K.,  &  von  Hofsten,  C.  (2003).  Peripheral  vision  effects  on  spatial 
orientation  driven  by  focused  depth.  Presented  at  a  poster  session  of  the  47^^  Annual  Meeting  of 
the  Human  Factors  and  Ergonomics  Society.  Denver,  CO. 

“Enhanced  support  of  pilot  spatial  orientation  (SO)  could  be  achieved  by  presenting 
flight-adapted  visual  flow  with  conformal  horizon  in  the  peripheral  visual  field.  This 
could  be  implemented  on. .  .HUDs. .  .or. . .HMDs. ...  The  [experimental]  results  indicate 
that  focused  depth  drives  the  effects  of  peripheral  visual  flow  on  SO. 

Fadden,  S.,  Ververs,  P.  M.,  &  Wickens,  C.  D.  (2001).  Pathway  HUDS:  Are  they  viable? 
Human  Factors  43, 173-193. 

Forbes,  T.W.  (1946).  Auditory  signals  for  instrument  flying.  Journal  of  the  Aeronautical 
Sciences,  May,  255-258. 

Tones  used  to  augment  visual  attitude  displays  and  directions. 

Gallimore,  J.J.,  Brannon,  N.G.,  Patterson,  F.R.,  &  Nalepka,  J.P.  (1999).  Effects  of  FOV  and 
aircraft  bank  on  pilot  head  movement  and  reversal  errors  during  simulated  flight.  Aviation, 

Space,  and  Environmental  Medicine,  70(12),  1152-1160. 

“Investigated  the  effects  of  field  of  view  (FOV)  and  aircraft  bank  on  pilot  head  move¬ 
ment  and  reversal  error  during  simulated  flight.  It  was  hypothesized  that  as  FOV 
decreased,  there  would  be  a  significant  reduction  in  Opto-Kinetic  Cervical  Reflex 
(OKCR)-induced  head  movement.  Reduced  FOV  was  also  hypothesized  to  increase 
control  reversal  errors.  To  test  this  hypothesis  12  US  military  pilots  completed  simulated 
flight  tasks  in  a  stationary  dome.  Head  tilt,  pitch,  and  yaw  were  recorded  as  a  function  of 
aircraft  bank  and  FOV  (40, 60,  and  100  degrees  circular).  The  number  of  control  reversal 
errors  was  analyzed  to  investigate  signs  of  spatial  disorientation.  Results  indicated  that 
during  visual  meteorological  condition  (VMC)  maneuvers  pilots  exhibited  significant 
OKCR;  however  there  were  no  significant  differences  among  the  3  levels  of  FOV.  FOV 
significantly  affected  head  pitch  movements  vmder  VMC  and  instrument  meteorological 
conditions.  Pilots  yawed  their  heads  in  the  direction  of  aircraft  bank  under  VMC.  Pilots 
committed  22  reversal  errors  out  of  72  trials  (30.55%).  The  magnitude  of  the  error  was 
largest  in  the  40  degree  FOV  condition. 
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Gallimore,  J.J.,  &  Liggett,  K.K.  (2000).  Implications  of  spatial  sensory  reflex  research  on 
primary  flight  display  design.  San  Antonio,  TX:  Spatial  Disorientation  Symposium. 

OKCR  occurs  in  VMC  (when  the  horizon  is  visible),  not  in  IMC.  Apparently,  it  is  due  to 
the  reflexive  response  to  keqj  the  horizon  stable  within  the  visual  field.  Head  tilt  fi'om 
OKCR  is  likely  to  put  the  pilot’s  eyes  outside  the  HUD  eye  box.  Frame  of  reference  k 
dictated  by  the  pilots’  attentional  activities.  Implications  for  transitions  between  different 
firames  of  reference  are  pertinent  to  HMD  symbology;  traditional  symbology  should  not 
be  used. 

Gallimore,  J.J.,  Patterson,  F.R.,  Brannon,  N.G.,  &  Nalepka,  J.P.  (2000),  The  opto-kinetic 
cervical  reflex  during  formation  flight.  Aviation,  Space,  and  Environmental  Medicine,  71(S), 
812-821. 

“Weather  formation  flight  is  a  difficult  task  prone  to  episodes  of  spatial  disorientation. 
Therefore,  investigation  of  sensory  reflexes  under  these  conditions  is  critical.  Recent 
studies  have  shown  that  the  opto-kinetic  cervical  reflex  (OKCR)  occurs  during  VMC 
flight  conditions  and  serves  to  establish  the  horizon  retinal  image  as  a  stabilized  primary 
visual-spatial  cue.  The  purpose  of  this  research  was  to  investigate  the  OKCR  and  field  of 
view  (FOV)  during  formation  flight  under  VMC  and  IMC.  There  were  2  experiments 
conducted  in  which  a  total  of  26  pilots  completed  simulated  flight  tasks  in  a  stationary 
dome.  Head  tilt  was  examined  m  a  function  of  aircraft  bank  with  imrestricted  FOV  in 
Experiment  I,  Experiment  n  examined  head  tilt  under  three  FOV  conditions  (40  degrees, 
60  degrees,  and  100  approximately  circular).  During  VMC  maneuvers  pilots  exhibited 
significant  OKCR.  There  were  no  differences  in  head  tilt  between  Solo  Figure  8  and 
Formation  Figure  8  VMC  conditions.  Pilots  did  not  tilt  their  heads  under  IMC  Formation 
Flight.  FOV  did  not  significantly  affect  the  OKCR.  Pilots  exhibit  the  OKCR  during 
formation  and  solo  VMC  tasks.  However,  the  OKCR  is  reduced  when  compared  with 
low  level  navigation  teks,  indicating  a  difference  in  the  visual  cues  between  tasks.  Pilots 
do  not  exhibit  OKCR  during  IMC  fli^t;  therefore,  the  OKCR  will  have  an  impact  on 
formation  flights  in  and  out  of  clouds  leading  to  sensory  conflicts  caused  by  repeated 
realignment  of  visual  and  vestibular  systems.” 

Gawron,  V,,  &  Knotts,  L.  (1984),  A  preliminary  flight  evaluation  of  the  peripheral  vision 
display  using  the  NT-33  A  aircraft.  Proceedings  of  the  Annual  Meeting  of  the  Human 

Factors  Society.  Santa  Monica,  CA:  Human  Factors  Society,  539-541, 

Gillingham,  K.K.  (1992).  The  spatial  disorientation  problem  in  the  United  States  Air  Force. 
Journal  of  Vestibular  Research.  2,297-306. 

“Spatial  disorientation  (SD)  in  flight  wastes  hundreds  of  millions  of  dollars  worth  of 
defense  capability  annually  and  continues  to  kill  air-crew.  SD  results  primarily  from 
inadequacies  of  human  visual  and  vestibular  sensory  sj^tems  in  the  flying  environment; 
but  other  factors,  such  as  task  saturation  and  distraction,  precipitate  it.  The  United  States 
Air  Force  is  conducting  a  three-pronged  research  and  development  effort  to  solve  the  SD 
problem.  We  are  attempting  1)  to  elucidate  further  the  mechanisms  of  visual  and  vesti¬ 
bular  orientation  and  disorientation,  2)  to  develop  ground-based  and  inflight  training 
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methods  for  demonstrating  to  pilots  the  potential  for  SD  and  the  means  of  coping  with  it, 
and  3)  to  conceive  and  evaluate  new  ways  to  display  flight  control  and  performance 
information  so  that  pilots  can  maintain  accurate  spatial  orientation.” 

Gillingham,  K.K.,  &  Previc,  F.H.  (1993).  Spatial  orientation  in  flight  (AL-TR-1993-0022). 

Wright-Patterson  AFB,  OH:  Air  Force  Armstrong  Laboratories. 

Gilson,  R.D.,  Ventola,  R.W.,  &  Fenton,  R.E.  (1975).  A  kinesthetic-tactual  display  for  stall 

deterreilce.  In  Eleventh  Annual  Conference  on  Manual  Control  (NASA  TM  X-62,464;  under 

FAA  contract  DOT-FA74WA-3515).  Moffett  Field,  CA:  Ames  Research  Center,  440-451 

Describes  a  device  for  the  yoke  or  stick  of  an  aircraft  that  pushes  into  the  hand  to  indicate 
AOA. 

Goldberg,  M.E.,  Eggers,  H.M.,  &  Gouras,  P.  (1991).  The  ocular  motor  system.  In  E.R. 

Kandel,  J.H.  Schwartz,  &  T.M.  Jessell  (Eds.),  Principles  of  neural  science.  Norwalk,  CT: 

Appleton  &  Lange,  660-677. 

Gomez,  G.  (2002).  Spatial  Disorientation:  Something  old  &  something  new.  In  Essays  & 

Articles.  Found  with  Google  search  at:  http://www.isamindia.org/essays/cme_spatial.shtml 

Excellent  historical  overview  and  statistics.  Vision  gives  90%  of  the  input  to  the  brain 
for  orientation.  “The  other  senses  play  a  supporting  role  in  correct  perception  of  aireraft 
orientation.”  “Visual  dominance  falls  into  two  categories:  the  congenital  type,  in  which 
ambient  vision  provides  dominant  orientation  cues  through  natural  neural  coimections 
and  functions,  and  the  acquired  type,  in  which  orientation  cues  are  gleaned  through  focal 
vision  and  are  integrated  as  a  result  of  training  and  experience  into  an  orientation  percept. 
This  complex  skill  must  be  developed  through  training  and  maintained  through  practice, 
and  its  fi*agility  is  one  of  the  factors  that  make  spatial  disorientation  such  a  hazard.” 
“Visual  cues  from  instruments  utilize  focal  rather  than  the  ambient  mode  of  vision  and 
therefore  place  greater  demands  on  central  processing  resources.”  “A  pilot’s  radio 
transmission  that  the  aircraft  controls  are  malfunctioning  should  not,  therefore,  be  taken 
as  conclusive  evidence  that  a  control  malfunction  caused  a  mishap:  spatial  disorientation 
could  have  been  the  real  cause.”  “The  basic  concept  of  SD  containment  is  to  convert  all 
Type  I  SD  to  Type  II  SD  and  prevent  a  Type  II  SD  from  becoming  a  Type  HI  SD.”  The 
author  also  presents  an  SD  Threat  Checklist,  which  includes:  “Flight  over  featureless  ter¬ 
rain”  (e.g.,  ocean  or  desert);  and  aircrew  factors,  such  as  “high  arousal  and  anxiety  which 
increase  susceptibility  to  SD,”  drugs  which  impair  a  pilot’s  “ability  to  suppress  nystag¬ 
mus,”  and  fatigue  or  task  overload. 

Gopher,  D.,  Weil,  M.,  &  Barakeit,  T.  (1994).  Transfer  of  skill  from  a  computer  game  to  flight. 

Human  Factors,  36,  387-405. 

Guedry,  F.E.  (1992).  Perception  of  motion  and  position  relative  to  the  earth:  An  overview. 

Annals  of  the  New  York  Academy  of  Sciences,  May  656, 315-328.  Found  at:  http://www. 

annalsnyas.org/cgi/content/abstract/65 6/1/315. 
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“Results  of  the  five  experiments  are  consistent  with  the  following  generalizations.  Canal- 
mediated  turn  perception  (pitch,  roll,  or  yaw)  in  earth-horizontal  or  earth-vertical  plane,  is 
suppressed  in  toect  relationship  to  the  magnitude  of  a  linear  acceleration  vector  lying  in 
the  plane  of  a  responding  canal  when  the  magnitude  of  the  linear  vector  is  constant  or 
increasing  and  when  its  direction  is  either  fixed  or  rotating  in  the  same  direction  as  the 
concomitant  canal  signal.  Canal-mediated  turn  perception  (pitch,  roll,  or  yaw)  is  not  sup¬ 
pressed  by  a  coplanar  linear  vector  that  is  counter-rotating  relative  to  the  canal  signal. 
Change  in  perceived  attitude  (pitch,  roll,  or  yaw)  is  very  sluggish  in  the  absence  of  con¬ 
cordant  canal  information;  attitude  change  may  not  be  an  immediate  otolith-mediated 
perceptual  event  but  a  slowly  developing  perception  dependent  upon  cognitive  apprecia¬ 
tion  of  an  immediate  otolith  angular  position  signal.  Otohth  phasic  neural  units,  unrein¬ 
forced  by  appropriate  canal  signals,  may  contribute  more  to  a  brief  linear  velocity  com¬ 
ponent  in  perception  than  to  rate  of  attitude  change.  Otolith-mediated  attitude  perception 
within  a  given  earth-vertical  plane  can  be  distorted  by  strong  coplanar  angular  velocity 
canal  information.  Once  distorted,  return  to  veridical  attitude  perception  can  be  gradual 
because,  in  fire  absence  of  complimentary  canal  or  visual  information,  recovery  is 
dependent  upon  relatively  slow  cognitive  appreciation  of  a  prevailing  otolith  position 
signal.  Several  attractive  hypotheses  relating  to  the  dynamics  of  attitude  perception  can 
only  be  tested  by  substantially  more  data  on  the  dynamics  of  spatial  orientation  percep¬ 
tion.  Most  of  our  objectives  caimot  be  achieved  without  models  that  yield  valid  predic¬ 
tion  of  the  dynamics  of  spatial  orientation  perception.  All  of  the  observations  in  these 
experiments  were  carried  out  in  darkness,  or,  in  the  simulated  catapult  experiment,  with¬ 
out  external  visual  reference.  Various  fomis  of  visual  information  will  change  the 
dynamics  of  spatial  orientation  perception.  My  discussion  has  been  limited  to  considera¬ 
tion  of  the  vestibular  system,  as  though  the  canal  and  otolith  sptems  completely  con¬ 
trolled  the  dynamics  of  spatial  orientation  perceptions.  Obviously  other  partners  in  the 
dynamics  of  postural  control,  including  vision,  proprioception,  and  expectation,  must  be 
included  in  tWs  challenging  field  of  research.  Dedication  to  stereotyped  ideas  about 
objectivity  in  the  20th  century  has  hindered  advancement  of  knowledge  on  the  dynamics 
of  spatial  orientation  perception  relative  to  rate  of  progress  achieved  by  several  scientists 
of  the  18th  and  19th  centuries,  who  provided  word  pictures  of  perceived  motions  and  tilts 
along  with  descriptions  of  the  motions  that  engaidered  the  pictures.” 
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Highlights  the  SD  problem  and  recent  approaches  to  training  and  displays.  “For  the 
USAF  alone,  this  [SD  mishap  rate]  Mds  up  to  an  average  of  $140  million  annually.” 
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This  article  suggests  a  general  model  and  formulas  for  signal  processing  in  pursuit  and 
compensatory  tracking  tasks,  which  are  helpful  for  explaining  observed  data,  but  are  not 
as  good  at  predicting  performance. 
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(2003).  Survey  of  spatial  disorientation  in  military  pilots  and  navigators.  Aviation,  Space,  and 
Environmental  Medicine,  74(9),  957-965. 

“752  UK  aviators  answered  an  SD  survey  that  showed  92%  of  respondents  experienced 
the  Leans,  the  most  firequently  experienced  SD  event  fi'om  the  survey.  Next  most  fi'e- 
quent  was  loss  of  horizon  due  to  atmospheric  conditions  (82%),  then  misleading  altitude 
cues  (79%),  sloping  horizon  (75%),  and  distraction-induced  SD  (66%).  The  researchers 
correlated  SD  experiences  with  training  (ground  and  in-flight)  and  found  a  significant 
correlation  between  training  and  SD  recognition  and  severity.  The  researchers  concluded 
that  in-flight  training  was  most  beneficial  and  should  be  tailored  to  the  specific  aircraft 
and  circumstances  (e.g.,  use  of  NVGs).” 
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K.R.Boff,  L.Kaufman,  &  J.P.  Thomas  (Eds.),  Handbook  of  Perception  and  Human  Performance 
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Visual-vestibular  interactions,  dampening  effects,  and  thresholds  of  detectable  motions. 
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and  cognitive  background. 

Human  signal  processing  bottlenecks  and  an  assertion:  “Generally  our  eyes  are  drawn  to 
moving  shapes,  then  complex,  different,  and  colorful  objects.” 

Ivanov,  A.I.,  &  Lapa,  V.V.  (2002).  The  effect  of  mobility  of  the  information  field  of  the 
helmet-mounted  indicator  on  pilot’s  efficiency.  Aviakosm.  Ekolog.  Med.,  55(4),  37-40. 

“Results  of  an  experimental  investigation  of  pilot’s  efficiency  with  the  use  of  a  hehnet- 
mounted  indicator  (HMI)  are  reported.  Displacement  of  the  HMI  information  field  in 
consequence  of  head  movements  in  search  for  external  objects  and  target  acquisition  was 
shown  to  impact  reliability  of  spatial  orientation  and  cause  difficulties  during  construc¬ 
tion  and  realization  of  aircraft  control  movements.  These  difficulties  are  ascribed  to 
impairment  of  functioning  of  the  human  ‘body  scheme’  (internal  systems  of  coordinates) 
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and  modification  of  the  geometric  relations  of  the  true  and  HMI  displayed  horizon.  The 
conclusion  is  that  to  improve  efficiency  of  pilots  using  HMI  designed  should  be  a  device 
which  will  align  the  HMI  information  field  with  the  longitudinal  cabin  axis  after  each 
head  movement.” 

Jia,  H.B.,  Yu,  L,S.,  Bi,  H.Z.,  Wang,  K.N.,  Liu,  Z.,  &  Xie,  S.J.  (2001).  Ground  simulation  of 
the  G-excess  illusion.  Space  Medicine  and  Medical  Engineering,  14(2),  88-91 . 

“To  observe  the  subjects’  perception  of  orientation  following  certain  head  movements  or 
change  of  simulator  cab  attitude  in  hypergravity  (HG),  and  assess  the  feasibility  of  simu¬ 
lating  G-excess  illusion  on  the  ground  by  a  centrifuge-like  Spatial  Disorientation  (SD) 
simulator.  1.6  G  force  field  was  generated  by  planetary  rotation  of  the  simulator.  Per¬ 
ception  of  orientation  of  the  cab  were  collected  fi’om  twelve  male  pilots’  report  following 
their  heads  pitch  movements  in  pitch  plane  or  cab  attitude  changes  in  roll  plane  under  1 .6 
G.  While  making  a  pitch-up  head  movement,  the  pilots  experienced  a  63.8  degrees  +/- 
48.3  degrees  pitch-up  attitude  change  of  the  cab  in  pitch  plane,  and  when  the  cab  was 
tilted  left  20  degrees,  pilots  experienced  a  tilt-left  perception  of  48.6  degrees  +/-  39.4 
degrees  in  roll  plane.  Althou^  there’s  strong  Coriolis  effects  onboard  the  SD  simulator 
under  1.6  G,  most  pilots  experienced  the  G-excess  illusion.  It  demonstrated  that  it  was 
feasible  to  use  the  centrifuge-like  device  to  generate  this  kind  of  illusion  on  the  groimd.” 
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“Electrotactile  (electrocutaneous)  stimulation  at  currents  greater  than  sensation  threshold 
causes  sensory  Captation,  which  temporarily  raises  the  sensation  threshold  and  reduces 
the  perceived  magnitude  of  stimulation.  After  15  min[utes]  of  moderately  intense  expo¬ 
sure  to  a  conditioning  stimulus  (10  s[econds]  on,  10  s[econds]  off),  the  sensation  thresh¬ 
old  elevation  for  seven  observers  was  60-270%,  depending  on  the  current,  fi'equency,  and 
number  of  pulses  in  the  burst  structure  of  the  conditioning  stimulus.  Increases  in  any  of 
these  parameters  increased  the  sensation  threshold  elevation.  Ad^tation  and  recovery 
were  each  complete  in  approximately  15  min[utes].” 

Kandel,  E.R.  (1991).  Perception  of  motion,  depth,  and  form.  In  E.R.  Kandel,  J.H.  Schwartz,  & 
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and  Environmental  Medicine,  49,  1080-1086. 

“Spatial  disorientation  (SD)  refers  to  an  incorrect  self-appraisal  of  the  attitude  or  motion 
of  the  pilot  and  his  aircraft  with  respect  to  the  earth.  This  paper  defines  elements  of  SD 
problems  as  encountered  in  general  civil  aviation.  Accident  reports  made  by  the  National 
Transportation  Safety  Board  for  a  recent  6-year  period  were  reviewed.  Statistical  com¬ 
putations  were  made  relating  SD  to  fatal  accidents.  Small  fixed-wing  aircraft  (under 
12,500  lb)  accounted  for  97.3  percent  of  all  SD  accidents.  Inclement  weather  was  associ¬ 
ated  with  42  percent  of  all  fatal  accidents,  and  SD  was  a  cause  or  factor  in  35.6  percent  of 
these  cases.  Non-instrument-rated  pilots  were  involved  in  84.7  percent  of  SD  weather- 
involved  accidents.  These  and  other  data  attest  to  the  importance  of  this  psychophysi- 
ological  phenomenon  (SD)  in  flight  safety.  Suggestions  are  made  of  ways  to  improve 
pilots’  awareness  and  understanding  of  this  problem.” 

Knapp,  C.J.,  &  Johnson,  R.  (1996).  F-16  Class  A  mishaps  in  the  U.S.  Air  Force,  1975-93. 
Aviation,  Space,  and  Environmental  Medicine,  67(Z),  111-1  %2>. 

“All  USAF  F-16  fighter  Class  A  (major)  aircraft  mishaps  from  1975-93  were  analyzed; 
using  records  firom  the  U.S.  Air  Force  Safety  Agency  (AFSA).  There  were  190  Class  A 
mishaps  involving  204  F-16’s  and  217  aircrews  during  this  19-yr  period.  The  overall 
Class  A  rate  was  5.09  per  100,000  flight  hours,  more  than  double  the  overall  USAF  rate. 
The  mishaps  are  categorized  by  year,  month,  time  of  day  and  model  of  aircraft  in  relation 
to  mishap  causes  as  determined  and  reported  by  AFSA.  Formation  position,  phase  of 
flight  and  primary  cause  of  the  mishap  indicate  that  maneuvering,  cruise  and  low-level 
phases  account  for  the  majority  of  the  mishaps  (71%),  with  air-to-air  engagements  asso¬ 
ciated  with  a  higher  proportion  of  pilot  error  (71%)  than  air-to-groxmd  (49%).  Engme 
failure  was  the  number  one  cause  of  mishaps  (35%),  and  collision  with  the  groimd  the 
next  most  fi'equent  (24%).  Pilot  error  was  determined  as  causative  in  55%  of  all  the  mis¬ 
haps.  Pilot  error  was  often  associated  with  other  non-pilot  related  causes.  Channelized 
attention,  loss  of  situational  awareness  and  spatial  disorientation  accotmted  for  approxi¬ 
mately  30%  of  the  total  pilot  error  causes  foimd.  Pilot  demographics,  flight  hour/sortie 
profiles,  and  aircrew  injuries  are  also  listed.  Fatalities  occurred  in  27%  of  the  mishaps, 
with  97%  of  those  involving  pilot  errors.” 
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“Researchers  examined  the  psychological  aspects  of  pilot  spatial  orientation  to  aircraft 
attitude.  Study  participants  completed  questionnaires  and  made  drawings  of  view-from- 
the-ground  and  aircraft  fi'ont  window  and  of  attitude  indicators,  then  made  flights  in  a 
simulator  or  aircraft  with  unknown  pitch  attitude.  Analysis  of  data  indicates  that  pilots 
favored  a  view-from-the-ground  indicator.  The  main  drawback  to  the  view-from-the- 
aircraft  indicator  was  mobility  of  image.  The  study  also  examined  spatial-orientation 
techniques  pilots  use  in  different  flight  ph^es  and  identified  seven  factors  in  effective 
spatial-orientation  techniques.  Other  components  of  the  study  included  a  comparison  of 
the  manipulative  capability  of  pilots  and  nonprofessional  participants  and  pilot  recall  of  a 
set  of  images  from  long-term  and  operational  memory.” 
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“This  report  is  the  final  report  of  the  first  phase  of  a  study  of  head  up  displays  (HUD) 
symbology  conducted  for  DGAC/SFACT,  in  the  frame  of  its  pre  normative  research  pro¬ 
gram  for  civil  aviation  safety.  This  report  describes  the  main  steps  and  results  of  the 
study: 

1 .  Comparative  analysis  of  the  JAA  and  FAA  regulations  concerning  HUD  in  general, 
and  more  especially,  the  specifications  about  their  symbology 

2.  State  of  the  interpretation  of  the  existing  regulations,  through  the  interviews  of 
certification  experts 

3.  Comparative  analysis  of  the  symbologies  of  existing  HUD  products  and  comparison 
Avith  the  requirements  of  the  regulation 

4.  Current  state  of  the  operational  use  of  HUD,  tiirough  the  available  safety  reporting 
sj^tems  and  interviews  with  the  users 

5.  Synthesis  of  the  lack  of  the  regulation  in  regard  with  the  needs  of  the  certification 
experts  and  with  the  operational  difficulties  encountered  by  the  users.” 
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“Spatial  disorientation  (SD)  refers  to  pilots’  inability  to  accurately  interpret  the  attitude  of 
their  aircraft  with  respect  to  Earth.  Unfortunately,  SD  statistics  have  held  constant  tor  the 
past  few  decades,  through  the  transition  Ifrom  the  head-down  attitude  indicator  (AI)  to  the 
head-up  display  (HUD)  as  the  attitude  instrument.  The  newest  attitude-indicating  device 
to  find  its  way  into  military  cockpits  is  the  helmet-mounted  display  (HMD).  HMDs  were 
initially  introduced  into  the  cockpit  to  enhance  target  location  and  weapon-pointing,  but 
there  is  currently  an  effort  to  make  HMDs  attitude  reference  displays  so  pilots  need  not 
go  head-down  to  obtain  attitude  information.  However,  unintuitive  information  or  inap¬ 
propriate  implementation  of  on-boresight  attitude  symbology  on  the  HMD  may  contrib¬ 
ute  to  the  SD  problem.  The  occurrence  of  control  reversal  errors  (CREs)  during  unusual 
attitude  recovery  tasks  when  using  an  HMD  to  provide  attitude  information  was  investi¬ 
gated.  The  effect  of  such  errors  was  evaluated  in  terms  of  altitude  changes  during  recov¬ 
ery  and  time  to  recover.  There  were  12  pilot-subjects  who  completed  8  unusual  attitude 
recovery  tasks.  Results  showed  that  CREs  did  occur,  and  there  was  a  significant  negative 
effect  of  these  errors  on  absolute  altitude  change,  but  not  on  total  recovery  time.  Results 
failed  to  show  a  decrease  in  the  number  of  CREs  occurring  when  using  the  HMD  as 
compared  with  data  from  other  studies  that  used  an  AI  or  a  HUD.  Results  suggest  that 
new  HMD  attitude  symbology  needs  to  be  designed  to  help  reduce  CREs  and,  perhaps, 
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“Spatial  disorientation  (SD)  continues  to  be  a  contributing  factor  to  a  fairly  constant  pro¬ 
portion  of  military  aircraft  accidents.  The  United  States  Air  Force  (US  AF)  fielded  a  new 
accident  investigation  reporting  form  in  July  1989,  which  for  the  first  time  specified  SD 
Type  1,  Type  2,  and  Type  3  as  possible  causes  of  aircraft  accidents.  Of  a  total  of  91 
major  accidents  that  occurred  over  the2-year  period  beginning  in  October  1989,  SD  con¬ 
tributed  significantly  to  13  (14  percent).  Although  this  percentage  is  higher  than  that 
reported  in  previous  studies,  the  actual  rate  of  SD  accidents  per  100,000  flying  hours 
(.1843)  is  lower  than  previously  reported.  Type  1  SD  was  the  cause  of  all  13  accidents;  9 
of  the  13  were  fatal;  6  occurred  in  ni^t  or  instrument  meteorological  conditions  (IMC) 
conditions;  and  1 1  involved  cockpit  attention  problems,  such  as  inattention,  distraction, 
or  channelized  attention.  Pilot  inexperience  did  not  appear  to  be  a  factor:  average  total 
flying  time  for  the  13  pilots  was  1,687  hours.  Coding  for  SD  on  accident  investigation 
reporting  forms  was  not  consistent.  There  were  both  individual  differences  between 
flight  surgeons  and  pilots,  and  trends  in  reporting  overtime.  There  is,  however,  a  consen¬ 
sus  that  SD  represents  a  major  problem  in  military  aviation.  A  scientific  approach  to  this 
important  problem  would  be  facilitated  if  agreement  could  be  reached  on  definitional  and 
semantic  issues.” 

Lyons,  T.J.,  Ercoline,  W.R.,  Freeman,  J.E.,  &  Gillingham,  K.K.  (1994).  Classification 
problems  of  U.S.  Air  Force  spatial  disorientation  accidents,  1989-91.  Aviation,  Space,  and 
Environmental  Medicine,  65, 147-152. 

Statistics  for  SD  accidents  and  illusions,  but  inconsistent  reporting  makes  it  difficult  to 
state  the  magnitude  of  specific  issues. 

Macurdy,  J.T.  (1934);  Disorientation  and  vertigo,  with  special  reference  to  aviation.  British 
Journal  of  Psychology,  25, 42-54. 

Malinin,  L,  Ercoline,  W.R.,  Previc,  F.H.,  &  Malinina,  H.  (2000).  The  consequences  of  adding 
runway  symbology  to  the  head-up  display.  Presented  at  the  Spatial  Disorientation  Symiposium, 
15-17November,  2000,  San  Antonio,  TX.  Found  at:  http://www.spatiald.wpafb.af.miF 
Primary%20Flight%20Displays/malininr.pdf 

While  the  focus  of  this  presentation  was  on  CFIT  accident  prevention  using  HUD  sym¬ 
bols,  it  offers  a  taxonomy  of  a  pilot’s  processing  of  flight  symbology  to  maintain  spatial 
attitude  awareness  during  IFR  and  VFR  flight. 

McCarley,  J.S.,  Wickens,  C.D.,  Goh,  J.,  &  Horrey,  W.J.  (2002).  A  computational  model  of 
attention/situation  awareness.  In  Proceedings  of  the  46**  Annual  Meeting  of  the  Human  F actors 
and  Ergonomics  Society.  Santa  Monica,  CA:  HFES. 

“A  computational  model  of  attention  and  situation  awareness  (SA)  was  developed  and 
used  to  predict  pilot  errors. ...  The  model  incorporates  a  low-level  perception/attention 
module  and  a  hi^er-level  belief-updating  module.  Attentionai  scanmng  is  connoiieu  ny 
bottom-up  and  top-down  processes,  with  the  effectiveness  of  top-down  guidance  varying 
as  a  function  of  SA.  Information  sampled  by  the  low-level  module  is  fed  forward  to  the 
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higher-level  module  for  consolidation  within  a  working  memory  representation  of  the 
pilot’s  situation,  with  the  quality  of  this  representation  reflecting  the  pilot’s  level  of  SA. 
The  model  was  validated  by  comparing  its  predictions  to  the  behavior  of  pilots. . .. 

Results  indicate  that  the  model  successfully  predicts  the  improved  performance  associ¬ 
ated  with  display  augmentations,  and  provides  construct  validity  regarding  the  effects  of 
visibility,  distraction,  and  degraded  information  quality.” 

McGrath,  B.  J.  (2000).  Tactile  instrument  for  aviation  (Naval  Aerospace  Medical  Research 
Laboratory  Monograph  49).  Pensacola,  FL:  NAMRL. 

Meehan,  J.W.  (2001).  Advanced  display  technologies:  What  have  we  lost?  HSIAC  Gateway 
72(3),  13-14. 

Attentional  capture  and  fixation  as  sources  of  problems  related  to  LS  A  and  SD. 

Melzer,  J.E.,  &  Moffitt,  K.  (1997).  HMD  design  -  Putting  the  user  first.  In  J.E.  Melzer  and  K. 

Head  mounted  displays:  Designing  for  the  user.  New  York:  McGraw-Hill. 

Merryman,  R.F.K.,  &  Cacioppo,  A.J.  (1997).  The  optokinetic  cervical  reflex  in  pilots  of  high- 
performance  aircraft.  Aviation,  Space,  and  Environmental  Medicine,  68(f)),  479-487. 

“While  modifications  of  the  cockpit  attitude  displays  are  unlikely  to  occur  in  the  near 
future,  the  ease  by  which  the  symbology  of  new  displays  (i.e.,  helmet-mounted  displays) 
can  be  changed  makes  them  the  prime  candidates  for  ‘OKCR-fiiendly’  design  changes. 
Caution  must  be  taken,  however,  since  multiple  attitude  displays  of  differing  types  may 
prevent  resolution  or  add  to  spatial  disorientation.  Therefore,  all  attitude  displays  within 
the  vehicle  should  be  compatible  with  a  singular  frame  of  reference  and  this  frame  of  ref¬ 
erence  should  be  determined  experimentally  and  include  the  OKCR  response.” 

Milgram,  P.,  &  Colquhoun,  H.,  Jr.  (1999).  A  taxonomy  of  real  and  virtual  world  display 
integration.  In  Y.  Ohta  and  H.  Tamura  (Eds.),  Mixed  reality  -  merging  real  and  virtual  worlds. 
Berlin:  Springer-Verlag. 

Nall  (1999).  1999  Nall  Report:  Accident  trends  and  factors  for  1998.  Report  foxmd  at  http:// 
www.aopa.org/asf/publications/99nall.html. 

“Spatial  disorientation  occurs  when  a  pilot  is  deprived  of  visual  references  to  determine 
the  aircraft’s  orientation  in  three-dimensional  space.  Any  conditions  which  deprive  the 
pilot  of  natural,  visual  references  to  maintain  orientation,  such  as  clouds,  fog,  haze,  dark¬ 
ness,  or  terrain/sky  backgrounds  with  indistinct  contrast  (such  as  arctic  whiteout  or  clear, 
moonless  skies  over  water)  can  rapidly  bring  about  spatial  disorientation.  Without  a 
means  of  controlling  the  airplane  with  reference  to  the  earth’s  surface,  loss  of  control  is 
imminent.  The  only  preventive  measure  is  to  rely  on  references  based  on  the  aircraft’s 
instruments.  The  aircraft  must  be  adequately  equipped  and  maintained  and  the  pilot  must 
be  sufficiently  trained  to  fly  solely  by  reference  to  instmments. 
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“In  1998,  six  accidents  contained  specific  references  to  spatial  disorientation  in  the 
sequence  of  events  or  narrative  sections  of  their  reports.  This  number  is,  however,  what 
statisticians  call  a  “lower  bound”  on  the  true  number  of  accidents  in  which  spatial  disori¬ 
entation  WM  a  significant  factor.  The  conditions  surrounding  a  number  of  other  weather- 
related  accidents  suggest  that  spatial  disorientation  might  have  been  contributory  there  as 
well. 

“A  detailed  analysis  of  accidents  over  a  ten-year  period  (1987-1996)  with  an  emphasis  on 
spatial  disorientation  as  a  cause  or  significant  contributory  factor  reveals  a  much  higher 
involvement  of  this  factor  than  suggested  by  the  direct  references  in  the  1998  reports. 
During  this  period,  there  was  an  average  of  almost  37.6  accidents  per  year,  of  which  33.9 
were  fatal.  At  this  rate,  there  is  one  fatal  spatial  disorientation  accident  every  eleven 
days.  Over  90  percent  of  all  the  accidents  during  this  time  in  which  spatial  disorientation 
was  a  factor  resulted  in  fatalities. 

‘Typically,  these  accidents  are  suffered  by  noninstrument-rated  pilots  attempting  to  com¬ 
plete  VFR  flights  in  instrument  meteorological  conditions.  At  least  one  accident  in  1998, 
however,  occurred  when  an  experienced  instrument-rated  pilot  in  a  well-equipped  tur¬ 
bine-powered  airplane  became  disoriented  during  the  visual  portion  of  a  circhng  IFR 
approach.  In  this  case,  a  moonless  ni^t  exacerbate  the  weather  conditions.” 

Navathe,  P.D.,  &  Singh,  B.  (1994a).  Prevalence  of  spatial  disorientation  in  Indian  Air  Force 

aircrew.  Aviation,  Space,  and  Environmental  Medicine,  65(12),  1082-1085. 

The  1®*  Aero  Medical  Training  Centre  has  surveyed  aircrew  of  the  Indian  Air  Force  to 
determine  the  prevalence  of  Spatial  Disorientation  (SD),  as  related  to  aircraft  stream,  age, 
flying  experience,  geographical  location  and  other  operationally  significant  variables. 

The  reported  prevalence  of  SD  is  75%  among  fighter  aircrew,  64%  in  transport  aircrew, 
and  55%  in  helicopter  aircrew.  Whereas  the  prevalence  of  SD  does  not  appear  to  vary 
significantly  with  age  and  flying  experience,  it  is  higher  in  fully  operational  pilots  and  in 
pilots  returning  to  flight  duties  after  a  ground  tenure,  as  compared  to  ab-initio  and  type- 
converting  pilots. 

Navathe,  P.D.,  &  Singh,  B.  (1994b).  An  operational  definition  for  spatial  disorientation. 
Aviation,  Space,  and  Environmental  Medicine,  65(12),  1153-1155. 

Nelson,  W.T.,  Hettinger,  L.J.,  Cunningham,  J.A.,  Brickman,  B.J^  Haas,  M.W.,  & 
McKinley,  R.L.  (1998).  Effects  of  localized  auditory  information  on  visual  target  detection 

performance  using  a  helmet  moimted  display.  Human  Factors,  40, 452-460. 

Niall,  K.K.  (2002).  Laser  projection  veraus  a  CRT  display  in  the  visual  perception  of  aircraft 
aspect.  Human  Factors,  44iA),  630-643. 

“High-resolution  visual  displays  have  been  designed  for  fli^t  shnulation  so  that  observ¬ 
ers  may  judge  the  aspect  angle  of  aircraft  at  far  distances.  The  present  experiment  com- 
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pares  two  display  devices  as  untrained  observers  judge  the  spatial  orientation  of  two  tar¬ 
get  aircraft:  F-15  and  F-16  jets.  The  display  devices  are  a  prototype  direct-write 
microlaser  projector  and  an  SXGA-format  CRT  display.  Observers’  accuracy  of  aircraft 
identification  is  better  with  the  laser  projector,  and  recognition  response  times  are  faster. 
A  simple  rule  was  found  to  fit  the  observers’  response  times;  it  is  expressed  in  terms  of  a 
statistic  on  the  autocorrelation  of  black-and-white  silhouette  images  of  aircraft.  Observ¬ 
ers’  estimates  of  aspect  are  biased  by  the  laser  projector,  whereas  observers’  estimates  of 
aspect  are  accurate  on  average  with  the  SXGA  display.  This  bias  in  estimation  of  aspect 
may  be  attributable  to  variations  in  line  brightness  introduced  by  the  laser  projector. 
Actual  or  potential  applications  of  this  research  include  the  evaluation  of  high-resolution 
visual  displays  for  the  training  of  basic  fighter  maneuvers  with  military  j  et  pilots.” 

NTSB  (2002).  Aircraft  accident  brief:  Cessna  335,  N8354N  near  Hillsboro,  Missouri,  October 

16,  2000  (AAR-02-02,  adopted  6/5/02).  Washington,  D.C.;  author. 

Ocker,  W.C.,  &  Crane,  C.J.  (1932).  Blind  flight  in  theory  and  practice.  San  Antonio,  TX: 

Naylor. 

O’Hare,  D.,  &  Roscoe,  S.N.  (1991).  Flight  deck  performance:  The  human  factor.  Ames,  lA: 

Iowa  State  University  Press. 

Leans  and  head  tilt  to  false  (sensed)  horizon  —  different  from  OKCR,  since  IMC,  not 
VMC.  The  Leans  involves  the  tilting  of  the  head  orthogonal  to  the  “apparent”  horizon  as 
signaled  through  the  vestibular  system,  when  that  system  has  been  co-opted  by  the  illu¬ 
sion  (washout).  This  is  different  from  OKCR.  In  short,  the  head  will  always  automati- 
eally  orient  itself  orthogonal  to  the  inferred/perceived  horizon.  In  VMC,  this  is  explicit, 
and  accounts  for  the  OKCR  when  the  aircraft  rolls.  In  IMC  without  illusions,  the  appar¬ 
ent  horizon,  in  a  coordinated  turn,  is  parallel  with  the  wings  of  the  aircraft.  Hence  the 
OKCR  is  not  observed  when  the  airplane  banks  (Gallimore’s  finding).  After  washout, 
the  apparent  horizon  is  at  an  angle  with  the  true  horizon.  Hence  the  head  orients  to  this 
apparent  horizon  after  the  plane  stabilizes  (accounting  for  the  lefcs). 

Okada,  T.,  Grunfeld,  E.,  Shallo-Hoffmann,  J.,  &  Bronstein,  A.M.  (1999).  Vestibular 

perception  of  angular  velocity  in  normal  subjects  and  in  patients  with  congenital  nystagmus. 

Brain,  122{J),  1293-1303. 

“A  technique  is  described  for  the  assessment  of  vestibular  sensation.  The  two  main  goals 
of  the  study  were  (i)  to  compare  the  perception  of  angular  velocity  with  the  eye  velocity 
output  of  the  vestibulo-ocular  reflex  and  (ii)  to  study  vestibular  function  in  patients  with 
congenital  nystagmus;  this  was  needed  since  most  previous  studies,  based  on  eye  move¬ 
ment  recordings,  have  been  inconclusive.  Subjects  indicated  their  perceived  angular 
velocity  by  turning  by  hand  a  wheel  connected  to  a  tachometer.  The  vestibular  stimuli 
used  consisted  of  sudden  deceleration  from  rotation  at  a  constant  horizontal  velocity  of 
90°/s  (‘stopping’  responses).  Eye  movements  were  recorded  simultaneously  with  electro¬ 
oculography.  In  normal  subjects  the  perceived  angular  velocity  decayed  from  the 
moment  of  deceleration  in  an  exponential  fashion.  The  meantime  constant  of  sensation 
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decay  was  ~16  s.  Eye  movement  velocity  decayed  with  a  similar  exponential  trajectory 
(time  constant  16  s).  Congenital  nystagmus  patients  showed  markedly  shortened  vesti¬ 
bular  sensation  (mean  time  constant  7  s).  The  following  conclusions  can  be  drawn:  (i) 
the  similarity  of  the  eye  velocity  and  perceptual  responses  suggests  that  these  two  systems 
receive  a  vestibular  signal  which  has  been  similarly  processed;  (ii)  the  time  constant  of 
the  responses  indicates  that  this  vestibular  signal  probably  originates  in  the  same  brain¬ 
stem  ‘velocity  storage’  integrator;  (iii)  the  technique  described  is  useful  for  clinical 
assessment  of  vestibular  function,  particularly  in  patients  with  ocular  motiUty  disordas; 
(iv)  patients  with  congenital  nystagmus  have  short  vestibular  time  constants,  which  is 
probably  due  to  changes  induced  in  velocity  storage  processing  by  the  persistent  retinal 
image  motion  present  in  these  patients.” 

Otakeno,  S.,  Matthews,  RS.,  Folio,  L.,  Previc,  F.H.,  &  Lessard,  C.S.  (2002).  The  effects  of 
visual  scenes  on  roll  and  pitch  thresholds  in  pilots  versus  nonpilots.  Aviation,  Space,  and 

Environmental  Medicine,  7 3{2),9B-I0l. 

“Previous  studies  have  indicated  diat,  compared  with  non-pilots,  pilots  rely  more  on 
■  vision  than  ‘seat-of-the-pants’  sensations  when  presented  with  visual-vestibular  conflict. 
The  objective  of  this  study  was  to  evaluate  whether  pilots  and  non-pilots  differ  in  their 
thresholds  for  tilt  pCTception  while  viewing  visual  scenes  depicting  simulated  flight.  This 
study  was  conducted  in  the  Advanced  Spatial  Disorientation  Demonstrator  (ASDD)  at 
Brooks  AFB,  TX.  There  were  14  subjects  (7  pilots  and  7  nonpilots)  who  recorded  tilt 
detection  thresholds  in  pitch  and  roll  while  exposed  to  sub-threshold  movement  in  each 
axis.  During  each  test  run,  subjects  were  presented  with  computer-generated  visual 
scenes  depicting  accelerating  forward  fli^t  by  day  or  night,  and  a  blank  (control)  condi¬ 
tion.  The  only  significant  effect  detected  by  an  analysis  of  variance  (ANOVA)  was  that 
all  subjects  were  more  sensitive  to  tilt  in  roll  than  in  pitch  [F  (2,24)  =  18.96,  p  <  0.001]. 
Overall,  pilots  had  marginally  higher  tilt  detection  thresholds  compared  with  non-pilots 
(p  =  0.055),  but  the  type  of  visual  scene  had  no  significant  effect  on  thresholds.  In  this 
study,  pilots  did  not  demonstrate  greater  visual  dominance  over  vestibular  and  proprio¬ 
ceptive  cues  than  non-pilots,  but  appeared  to  have  higher  pitch  and  roll  thresholds  over¬ 
all.  The  finding  of  significantly  lower  detection  thresholds  in  the  roll  axis  vs.  the  pitch 
axis  was  an  incidental  finding  for  bofii  subject  groups.” 

Pancratz,  D.J.,  Bomar,  J.B.,  Jr.,  &  I^ddin,  J.H.,  Jr.  (1994).  A  new  source  for  vestibular 
illusions  in  high  agility  aircraft.  Aviation,  Space,  and  Environmental  Medicine,  65(12),  1 130- 
1133. 

“The  enhanced  maneuverability  aircraft  of  the  future  will  expose  pilots  to  combinations 
of  conventional  translational  accelerations  as  well  as  extraordinary  angular  accelerations. 
This  flight  regime,  combined  with  the  intense  concentration  required  for  combat  maneu¬ 
vering,  will  make  motion-induced  illusions  more  perilous  than  in  existing  aircraft. 
Although  there  are  many  causes  for  disorientation,  theoretical  anal^is  indicates  two  in 
particular,  the  ‘G-excess’  and  ‘cross-coupling’  illusions,  may  be  invoked  by  a  new  and 
distinctly  different  stimulus.  These  two  illusions,  which  are  both  typically  induced  by 
motion  of  the  pilot’s  head,  may  in  addition  be  created  by  rotation  of  the  aircraft  with 
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respect  to  its  flight  path.  After  comparing  typical  pilot  head  movements  to  projected 
decoupled  angular  motion  capabilities  of  supermaneuverable  aircraft,  we  conclude  that 
the  potential  exists  for  G-excess  or  cross-coupling  illusions  in  a  high  agility  aircraft  inde¬ 
pendent  of  pilot  head  motion  with  respect  to  the  aircraft. 

Patterson,  F.R.,  Cacioppo,  A.J.,  Gallimore,  J.J.,  Hinman,  G.E.,  &  Nalepka,  J.P.  (1997). 
Aviation  spatial  orientation  in  relationship  to  head  position  and  attitude  interpretation.  Aviation, 
Space,  and  Environmental  Medicine,  68(6),  463-71. 

Pilots  exhibit  OKCR  in  VMC,  not  in  IMC,  in  fixed-base  simulator.  “Conventional  wis¬ 
dom  describing  aviation  spatial  awareness  assumes  that  pilots  view  a  moving  horizon 
through  the  windscreen.  This  assumption  presupposes  head  alignment  with  the  cockpit 
‘Z’  [vertical]  axis  during  both  visual  (VMC)  and  instrument  (IMC)  maneuvers.  Even 
though  this  visual  paradigm  is  widely  accepted,  its  accuracy  has  not  been  verified.  The 
purpose  of  this  research  was  to  determine  if  a  visually  induced  neck  reflex  [OKCR] 
causes  pilots  to  align  their  heads  toward  the  horizon,  rather  than  the  cockpit  vertical  axis. 
Some  14  military  pilots  completed  two  simulated  flights  in  a  stationary  dome  simulator. 
The  flight  profile  consisted  of  five  separate  tasks,  four  of  which  evaluated  head  tilt  during 
exposure  to  unique  visual  conditions  and  one  examined  occurrences  of  disorientation 
during  unusual  attitude  recovery.  During  simulated  visual  flight  maneuvers,  pilots  tilted 
their  heads  toward  the  horizon  (p  <  0.0001).  Under  IMC,  pilots  maintained  head  align¬ 
ment  with  the  vertical  axis  of  the  aircraft.  During  VMC  maneuvers  pilots  reflexively  tilt 
their  heads  toward  the  horizon,  away  fi'om  the  Gz  axis  of  the  cockpit.  Presumably,  this 
behavior  stabilizes  the  retinal  image  of  the  horizon  (1  degree  visual-spatial  cue),  against 
which  peripheral  images  of  the  cockpit  (2  degrees  visual-spatial  cue)  appear  to  move. 
Spatial  disorientation,  airsickness,  and  control  reversal  error  may  be  related  to  shifts  in 
visual-vestibular  sensory  alignment  during  visual  transitions  between  VMC  (head  tilt) 
and  IMC  (Gz  head  stabilized)  conditions.  The  current  moving  horizon  attitude  displays 
lack  in  both  pictorial  reality  and  control  motion  compatibility.  Forced  spatial  reversal 
during  VMC  to  IMC  transition  may  contribute  to  disorientation,  reversal  error,  and  task 
saturation.” 

Pines,  M.  (2003).  The  mystery  of  smell:  The  vivid  world  of  odors.  In  Seeing,  hearing,  and 
smelling  the  world  (report  from  the  Howard  Hughes  Medical  Institute).  Found  at:  http://www. 
hhmi.org/senses/dl  lO.html 

“Smells. .  .retain  an  uncanny  power  to  move  us.”  The  article  also  describes  how  odor  and 
taste  signals  reach  the  brain. 

Previc,  F.  H.  (1998).  The  neuropsychology  of  3-D  space.  Psychological  Bulletin,  124, 123-164. 

Previc,  F.H.  (2000a).  Neurophysiological  guidelines  for  aircraft  control  stations.  IEEE 
Engineering  in  Medicine  and  Biology,  March/April,  81-87. 

Previc,  F.H.  (2000b).  In  search  of  ambient  vision.  San  Antonio,  TX:  Spatial  Disorientation 
Symposium.  Found  at:  http://www.spatiald.wpafb.af  mil/MechanismsStudies/previc.pdf 
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Defines  ambient  vision  and  explains  its  role  in  orientation  and  the  perception  of  motion. 

Previc,  F.H.,  &  Ercoline,  W.R  (1999).  The  “outside-in”  attitude  display  concept  revisited. 

The  International  Journal  of  Aviation  Psychology,  P(4),  377-401 . 

A  moving  aircraft  (outside-in)  attitude  display  has  been  shown  superior  to  the  widely 
used  inside-out  (moving  horizon)  attitude  display.  Novice  and  trained  pilots  both  per¬ 
form  better  with  the  outside-in  display  when  recovering  fi-om  unusual  attitudes.  Since  it 
has  proven  very  difficult  to  change  the  status  quo,  this  display  perspective  may  be  well 
suited  for  off-axis  HMD  applications,  UAV  control,  and  so-called  global  situation  dis¬ 
plays.  Since  the  outside-in  display  reduces  roll-reversal  errors,  the  authors  cite  statistics 
to  show  the  importance  of  reducing  such  errors:  “A  5%  to  10%  roll-reversal  error  in 
recovering  fi'om  unusual  attitudes  has  implications  for  military  and  civilian  aviation. 

Bank  illusions  are  reportedly  the  most  prevalent  form  of  spatial  disorientation  in 
pilots. .  .and  spatial  disorientation  remains  a  leading  cause  of  military  and  civilian  aircraft 
losses.  During  the  1980s,  for  instance,  spatial  disorientation  resulted  in  81  USAF  Class 
A  mishaps,  or  approximately  12%  of  all  Class  A  mishaps  and  nearly  25%  of  Class  A 
mishaps  attributable  to  operator-related  causes. . ..  Spatial  disorientation,  together  with 
loss  of  situational  awareness,  accounted  for  a  staggering  85%  of  all  USAF  operator- 
related  mishap  fatalities  during  this  same  period. . . .  Over  that  same  period,  112  U.S. 
Navy  Class  A  mishaps  (representing  about  15%  of  the  entire  Navy  total)  were  attributed 
to  either  possible  (61),  probable  (18),  or  definite  (33)  spatial  disorientation. ...  In  general 
aviation,  the  percentage  of  all  mishaps  due  to  spatial  disorientation  is  2%  to  2.5%,  but  the 
percentage  of  fatal  mishaps  due  to  disorientation  is  at  least  10%....  Roll-reversal  errors, 
in  particular,  are  believed  to  have  contributed  to  many  spatial  disorientation  mishaps  over 
the  years,  including  several  commercial  airline  disasters. .  .and  at  least  two  U.S. 

Navy. .  .and  two  USAF  crashes  in  1 996  and  1 997.  It  is  disturbing  that  in  advanced  air¬ 
craft  that  cost  tens  of  millions  of  dollars  or  more,  pilots  still  cannot  properly  orient  by 
means  of  their  attitude  instruments  when  disoriented  and  must  resort  to  the  well-known 
‘waffling’  with  the  stick  to  determine  the  correct  righting  direction  during  unusual-atti¬ 
tude  recoveries.  Not  surprisingly,  the  spatial  disorientation  mishap  rate  in  the  USAF  has 
not  fimdamentally  changed  over  the  past  three  decades.” 

Previc,  F.H.,  &  Ercoline,  W.R  (2001).  Trends  in  spatial  disorientation  research.  Aviation 

Space  and  Environmental  Medicine,  72( 1 1 ),  1 048- 1 050. 

A  60-year  retrospective  categorization  of  the  SD  literature  into  347  articles  into  altitude 
iiwtrumentation,  SD  incidence  and  SD  training.  A  dramatic  increase  in  SD  research 
began  in  the  early  90s. 

Raj,  A.K.,  Kass,  S.J.,  &  Periy,  J.F.  (2000).  Vibrotactile  displays  for  improving  spatial 

awareness.  Proceedings  of  the  Human  Factors  and  Ergonomics  Society  Annual  Meeting.  Santa 

Monica,  CA:  Human  Factors  Society,  181-184. 
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Reed,  R.R.  (2003).  Molecular  approaches  to  olfaction.  In  In  the  lab:  HHMI  investigators. 
Found  at  http://\vww.hhnii.org/research/mvestigators/reed.html.  Last  updated  February  4,  2003. 

“Chemosensation,  the  detection  of  chemicals  in  the  environment,  is  among  the  oldest  of 
the  sensory  systems. ...  These  external  stimuli  are  converted  into  nerve  impulses  and 
subsequent  cognitive  and  behavioral  responses.” 

Regan,  D.  (1995).  Spatial  orientation  in  aviation:  Visual  contributions.  Journal  of  Vestibular 
Research,  5{6'),A55-A1\. 

“Misjudgment  of  spatial  orientation  is  reported  to  be  an  important  factor  in  a  substantial 
proportion  of  aviation  accidents.  The  likelihood  of  such  misjudgment  is  particularly  high 
when  there  are  conflicting  interactions  between  labyrinthine  and  visual  signals.  In  a 
maneuvering  aircraft,  visual  information  is  often  a  crucial  factor  in  the  pilot’s  ability  to 
judge  the  aircraft’s  altitude  and  position  with  respect  to  external  and  terrain  features.  A 
body  of  research  evidence  supports  the  following  conceptual  framework.  (1)  The  human 
visual  system  processes  a  limited  number  of  visual  dimensions  almost  independently  of 
each  other  (for  example,  color,  luminance,  spatial  frequency,  orientation,  changing  size, 
motion  in  depth,  time  to  contact).  (2)  Each  of  these  selective  sensitivities  shows  consid¬ 
erable  inter-subject  variability.  (3)  Inter-subject  variability  is  far  from  perfectly  corre¬ 
lated  across  the  set  of  selective  sensitivities.  This  paper  discusses  the  relevance  of  this 
framework  for  predicting,  first  the  specific  situations  in  which  visual  judgments  are  most 
likely  to  occur  and,  second,  which  individual  pilots  are  most  likely  to  make  the  visual 
misjudgments.” 

Reising,  J.M.,  Liggett,  K.K.,  &  Munns,  R.C.  (1999).  Controls,  displays,  and  workplace 
design.  In  D.J.  Garland,  J.A.  Wise,  &  V.  D.  Hopkin  (Eds.),  Handbook  of  Aviation  Human 
Factors.  Mahwah,  NJ:  Lawrence  Erlbaum  Associates,  327-354. 

Display  concepts  for  unusual  attitude  recovery.  Crew  station  design  process  emphasizes 
modeling,  information  requirements,  evaluations,  and  traceability  used  in  an  iterative 
fashion. 

Rochlis,  J.L.,  &  Newman,  D.J.  (2000).  A  tactile  display  for  International  Space  Station  (ISS) 
extravehicular  activity  (EVA).  Aviation,  Space,  and  Environmental  Medicine,  77,571-8. 

“The  Tactor  Locator  System  (TLS)  is  a  non-intrusive,  intuitive  display  capable  of  con¬ 
veying  position  and  velocity  information  via  a  vibrotactile  stimulus  applied  to  the  sub¬ 
ject’s  torso  and  neck. . ..  The  TLS  was  designed  to  provide  somatosensory  cues  to  com¬ 
plement  the  visual  system. . 

Rouse,  W.B.,  &  Gopher,  D.  (1977).  Estimation  and  control  theory:  Application  to  modeling 
human  behavior.  Human  Factors,  7P(4),  315-329. 

While  not  specific  enough  for  our  SO/SD  modeling,  this  article  describes  formulas  for 
human-machine  system  models. 
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Roscoe,  S.N.  (1968).  Airborne  displays  for  flight  and  navigation.  Human  Factors,  10,  321-332. 

Roscoe,  S.N.  (2002).  Ergavionics:  Designing  the  job  of  flying  an  airplane.  International 

Journal  of  Aviation  Psychology,  22(4),  33 1  -339. 

Roscoe,  S.N.,  &  Williges,  R.C.  (1975).  Motion  relationships  in  aircraft  attitude  and  guidance 

displaj^:  A  flight  experiment.  Human  Factors,  17,314-3%! . 

Rosenzweig,  M.R.,  Breedlove,  S.M.,  &  Leiman,  A.L.  (2002).  Biological  Psychology  (3rd  ed.). 

Sunderland,  MA:  Sinauer  Associates,  Inc. 

Rupert,  A.H.  (2000).  Tactile  situation  awareness  system:  proprioceptive  prostheses  for  sensory 

deficiencies.  Aviation,  Space,  and  Environmental  Medicine,  77(9),  A92-99. 

“Pilots  and  astronauts  do  not  experience  spatial  disorientation  in  normal  day-to-day  ter¬ 
restrial  activities.  On  tiie  ground,  the  perception  of  position  and  motion  is  determined  by 
central  nervous  system  integration  of  concordant  and  redundant  information  from  multi¬ 
ple  sensory  channels  (somatosensory,  vestibular  and  visual)  which  collectively  yield 
veridical  percqjtions.  In  the  acceleration  environments  experienced  by  pilots  and  ^tro- 
nauts,  flie  somatosensory  and  vestibular  senses  frequently  present  false  but  concordant 
information  concerning  the  direction  of  gravity  or  down.  When  presented  with  conflict¬ 
ing  sensory  stimuli,  it  is  normal  for  pilots  and  astronauts  to  experience  episodes  of  disori¬ 
entation.  Visual  instruments  and  displays  developed  over  the  past  70  yr  have  not  solved 
the  problem.  A  simple  solution  to  maintain  spatial  orientation  is  to  provide  true  informa¬ 
tion  using  the  same  sensory  channels  we  use  so  successfully  on  Earth.  The  Tactile 
Situation  Awareness  System  (TSAS)  developed  by  NASA  and  the  U.S.  Navy  us^  a 
matrix  of  mechanical  tactile  stimulators  (tactors)  applied  on  the  torso  and  limbs  to  convey 
orientation  cues  (e.g.,  gravity  vector)  in  an  intuitive  fashion  to  the  skin.  A  series  of  in¬ 
flight  experiments  to  validate  and  test  a  variety  of  tactile  displays  and  concepts  has  been 
carried  out  in  both  helicopters  and  fixed  wing  aircraft.  Pilots  were  able  to  fly  complex 
maneuvers  with  no  instruments  or  outside  visual  references  (blindfolded)  with  less  than 
20  min  of  training.  Recovery  from  unusual  attitudes  solely  by  tactile  cues  was  trivial. 

Lab  tests  have  shown  the  TSAS  improves  performance  under  conditions  of  high  work¬ 
load.  When  orientation  information  is  presented  via  intuitive  tactile  displays  spatial  ori¬ 
entation  is  easily  maintained  in  altered  sensory  conditions  including  unusual  acceleration 
environments.” 

Schneider,  E.,  &  Glasauer,  S.  (date  unknown).  Temporal  summation  of  vestibular  afferents  in 

humans  affects  perceptional  threshold  of  rotation  around  the  yaw  axis.  Foxmd  on  1 1/6/2003  at: 

htq3://www.nefo.med.uni-muenchen.de/~eschneider/threshold/ 

“When  acoustical,  visual  and  somatosensorial  cues  are  lacking,  the  perception  of  whole 
body  rotation  about  the  yaw  axis  relies  on  the  stimulation  of  &e  horizontal  semicircular 
canals.  On  the  basis  of  their  mechanics,  which  can  be  described  by  a  second-order  dif¬ 
ferential  equation,  their  afferent  output  basically  reflects  angular  velocity.  Although 
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physiological  studies  of  these  afferents  have  not  established  a  mechanical  or  neural 
threshold,  humans  perceive  rotatory  movements  only  at  angular  velocities  above  a 
threshold  of  1  to  4  deg/sec. 

“The  experimental  data  in  the  literature  reveals  that  the  behavior  of  vestibular  thresholds 
cannot  be  described  with  only  the  mechanical  cupula  model.  Considerable  evidence  sug¬ 
gests  that  vestibular  thresholds  arise  in  a  more  central  region  of  the  nervous  system  so 
that  additional  components  are  added  to  their  behavior.  While  no  current  theoretical 
model  can  predict  this  more  complex  behavior  of  vestibular  thresholds,  it  is  well  known 
that  Zwislocki’s  theory  of  temporal  summation  can  be  applied  to  the  threshold  of  audi¬ 
bility  to  explain  its  time  variant  behavior  (Zwislocki,  1960). 

“This  prompted  us  to  investigate  whether  the  same  theory  could  be  applied  to  explain 
thresholds  for  detecting  rotation  around  the  yaw  axis.  Since  Zwislocki’s  theory  is  based 
on  the  assumption  of  an  exponential  decay  of  neural  excitation  we  used  stimuli  consisting 
of  pairs  of  rectangular  afferent  pulses  separated  by  time  intervals  of  1 50,  250,  and  350 
ms.  The  duration  of  one  pulse  was  250  ms.  To  obtain  such  afferent  responses,  the  tra¬ 
jectories  of  the  angular  velocities  had  to  be  computed  by  solving  the  inverse  differential 
equation  of  the  combined  cupula  and  chair  mechanics  and  adding  the  result  to  a  base 
velocity  of  5  deg/s.  The  mechanical  time  constants  used  were  5  s  and  5  ms  for  the  cupula 
and  200  ms  for  the  chair. 

“Four  subjects  participated  in  the  experiments.  Each  sat  on  a  rotating  chair  in  coriipiete 
darkness.  During  each  individual  session  the  first  of  the  two  pulses  remained  unchanged 
at  a  subliminal  amplitude.  The  thresholds  for  the  second  pulses  were  determined  for  the 
3  discrete  time  intervals  using  an  adaptive  psychophysical  procedure  (Benson,  1989). 

“If  a  time  constant  of  temporal  summation  between  300  and  1000  ms  is  assumed,  the 
threshold  shift  fi:om  the  stimulus  with  150  ms  pulse  interval  to  the  stimulus  with  350  ms 
pulse  interval  would  be  predicted  to  lie  within  the  range  of  2-4  dB.  However,  we  meas¬ 
ured  a  threshold  shift  of  about  8  dB.  This  is  much  more  than  a  model  consisting  of  one 
temporal  summing  leaking  integrator  can  explain. 

“When  we  used  a  third  order  instead  of  a  first  order  leaking  integrator  with  a  time  con¬ 
stant  of  190  ms  +/-  20  ms,  the  model’s  predictions  optimally  fit  our  experimental  data. 
This  value  was  almost  identical  to  the  time  constant  of 200  ms  measured  in  the  auditory 
pathway.  The  vestibular  pathway,  however,  seems  to  differ  from  the  auditory  pathway  in 
the  number  ofleaking  integrators  involved:  it  apparently  contains  three. 

“By  introducing  such  a  third  order  lag  with  a  transfer  function  of  H(s)  =  (T(s+l)-3  in  the 
vestibular  system,  it  is  possible  to  theoretically  explain  the  modification  of  the  Mulder 
product  found  empirically  by  Huang  and  Young  (Huang  &  Young,  1981).  The  simulated 
predictions  of  our  model  for  the  relation  between  angular  acceleration  a  and  time  to 
detect  tmin  are  very  similar  to  their  results,  which  they  estimated  with  the  equation  a  x 
pow(  tmin,  1 . 1 6 )  =  3 .26.” 
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Scott,  W.B.  (1999).  Automatic  GCAS:  You  can’t  fly  any  lower.  Aviation  Week  and  Space 
Technology,  150(5),  76-80.  > 

Sekular,  R.,  &  Blake,  R.  (2002).  Perception  (4*  ed.).  NY:  McGraw-Hill  Higher  Education. 

Self,  B.P.,  Breun,  M.,  Feldt,  B.,  Perry,  C.,  &  Ercoline,  W.R.  (2002).  Assessment  of  pilot 
performance  using  a  moving  horizon  (inside-out),  a  moving  aircraft  (outside-in),  and  an  arc- 
segmented  attitude  reference  display.  Presented  at  the  NATO  symposium.  Spatial  disorientation 
in  military  vehicles.  La  Caruna,  Spain:  NATO  Research  &  Technology  Agency. 

The  arc-segmented  attitude  display  proved  to  be  better  than  either  the  traditional  moving 
horizon  or  the  moving  aircraft  displays  for  unusual  attitude  recoveries  by  experienced  and 
novice  pilots. 

Sipes,  W.E.,  &  Lessard,  C.S.  (1999).  Spatial  disorientation:  A  survey  of  incidence.  InR.S. 
Jensen,  B.  Cox,  J.D.  Callister,  &  R.  Lavis  (Eds.),  Proceedings  of  the  Tenth  International 
Symposium  on  Aviation  Psychology.  Columbus,  OH:  The  Ohio  State  University.  ^ 

Sklar,  A.,  &  Sarter,  N.  (1999).  Good  vibrations:  Tactile  feedback  in  support  of  attention 

allocation  and  human  automation  coordination.  Human  Factors,  41, 543-552. 

Smith,  D.R,  Cacioppo,  A.J.,  &  Hinman,  G.E.,  Jr.  (1997).  Aviation  spatial  orientation  in 
relationship  to  head  position,  altitude  interpretation,  and  control.  Aviation,  Space,  and 
Environmental  Medicine,  68(6),  472-8. 

“Recently,  a  visually  driven  neck  reflex  was  identified  as  causing  head  tilt  toward  the 
horizon  during  VMC  flight.  If  this  is  the  case,  then  pilots  orient  about  a  fixed  rather  than 
moving  horizon,  implying  current  attitude  instruments  inaccurately  present  spatial  infor¬ 
mation.  The  purpose  of  this  study  was  to  determine  if  the  opto-kinetic  cervical  neck 
reflex  has  an  effect  dependent  on  passive  (autopilot)  or  active  control  of  the  aircraft. 
Further,  findings  could  help  determine  if  the  opto-kinetic  cervical  reflex  is  cha-acteristic 
of  other  flight  crewmembers. . ..  There  were  16  military  pilots  who  flew  two  13-miii[ute] 
VMC  lowdevel  routes  in  a  large  dome  flight  simulator.  Head  position  in  relation  to  air¬ 
craft  bank  angle  was  recorded  by  a  head  tracker  device.  During  one  low-level  route,  the 
pilot  had  a  supervisory  role  as  the  autopilot  flew  the  aircraft  (passive).  The  other  route 
was  flow  manually  by  the  pilot  (active). ...  Klots  consistently  tilted  the  head  to  maintain 
alignment  with  the  horizon.  Similar  head  tilt  angles  were  found  in  both  the  active  and 
passive  flight  phases.  However,  head  tilt  had  a  faster  onset  rate  in  the  passive  condi- 
'tion. . ..  Results  indicate  the  opto-kinetic  cervical  reflex  affects  pilots  while  actively  fly¬ 
ing  or  in  a  supervisory  role  as  the  autopilot  flies.  The  consistent  head  tilt  angles  in  both 
conditions  should  be  considered  in  attitude  indicator,  HUD,  and  HMD  designs.  Further, 
results  seem  to  indicate  that  non-pilot  flight  crewmembers  are  affected  by  the  opto¬ 
kinetic  cervical  reflex  which  should  be  considered  in  spatial  disorientation  and  aimick- 
ness  discussions.” 
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Snow,  M.P.,  Reising,  J.M.,  Liggett,  K.K.,  &  Barry,  T.P.  (1999).  Flying  complex  approaches 
using  a  head-up  display:  Effects  of  visibility  and  display  type.  In  10th  International  Aviation 
Psychology  Symposium.  Columbus,  OH:  Ohio  State  University. 

Advanced  head-up  and  head-down  displays  show  a  pathway-in-the  sky  and/or  a  synthetic 
terrain  image  for  pilot  orientation. 

Solomonow,  M.,  Lyman,  J.,  &  Freedy,  A.  (1977).  Electrotactile  two-point  discrimination  as  a 
function  of  frequency,  body  site,  laterality,  and  stimulation  codes.  Annals  of  Biomedical 
Engineering,  5, 47-60. 

“The  feasibility  of  frequency  modulated  two-point  discrimination  as  a  design  concept  for 
an  electrocutaneous  sensory  substitution  display  has  been  studied.  Three  stimulation 
techniques  were  tested  on  human  subjects:  spatial  stimulus,  temporal  stimulus,  and  fre¬ 
quency-on-frequency  stimulus.  The  frequency-on-frequency  technique  yielded  the  low¬ 
est  threshold  when  compared  to  the  temporal  and  spatial  techniques.  In  addition,  some  of 
the  characteristic  behavior  of  cutaneous  sensation  is  discussed  relating  two-point  dis¬ 
crimination  with  frequency,  body  sites,  and  stimulation  codes.  Implications  of  the  results 
for  clinical  applications  are  reviewed.” 

Solomonow,  M.,  Raplee,  L.,  &  Lyman,  J.  (1978).  Electrotactile  two-point  discrimination  as  a 
function  of  frequency,  pulse  width  and  pulse  time  delay.  Annals  of  Biomedical  Engineering,  6, 
117-125. 

“The  feasibility  of  frequency  modulated  two-point  discrimination  as  a  design  concept  for 
electrocutaneous  sensory  substitution  displays  has  been  investigated  further.  Two  new 
parameters  have  been  tested  on  human  subjects  for  their  effect  on  the  two-point  thresh¬ 
old:  pulse  width  and  pulse  time  delay  (phase  shift).  The  pulse  width  study  has  shown 
that  100-psec  pulses  resulted  in  the  lowest  threshold  for  spatial  stimulation  while  lO-psec 
pulses  yielded  the  lowest  threshold  for  temporal  stimulation.  It  was  also  shown  that  pulse 
phase  shifts  of  0  to  180°  result  in  different  threshold  values.  Phase  shifts  of  0  to  135° 
showed  slight  threshold  improvement.  The  180°  phase  shift  yielded  substantial 
improvement.” 

Spence,  C.  (2002).  Multisensory  attention  and  tactile  information-processing.  Behavioral  Brain 
Research,  135,  pp.  57-64. 

Stapleford,  R.L.  (1968).  Multimodality  pilot  model  for  visual  and  motion  cues.  Proceedings  of 
Fourth  Annual  NASA-University  Conference  on  Manual  Control  (NASA  SP-192).  Ann  Arbor, 
MI:  University  of  Michigan,  47-56. 

Estimates  of  semicircular  canal  thresholds  of  detection  for  pitch,  roll,  and  yaw  accelera¬ 
tions,  plus  formulas  for  converting  acceleration  times  into  sensed  velocities.  The  values 
given  are  3.2  deg/sec  for  roll,  2.6  deg/sec  for  pitch,  and  1 . 1  deg/sec  tor  yaw.  Also,  utncle 

sensing  of  linear  accelerations  is  discussed. 
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Taylor,  J.B.,  &  Kuchar,  J.K.  (2000).  Helmet-momited  display  symbology  for  terrain 
avoidance  during  low-level  maneuvers.  InternationalJournal  of  Aviation  Psychology,  10, 155- 
168. 

Taylor,  R.M.,  Bonner,  M.C.,  Dickson,  B.,  Howells,  H.,  Miller,  C.A.,  Milton,  N.,  Pleydell- 
Pearce,  K.,  Shadbolt,  N.,  Tennison,  J.,  &  Whitecross,  S.  (2002).  Cognitive  cockpit 
engineering:  Coupling  fimctional  state  assessment,  task  knowledge  management,  and  decision 
support  for  context-sensitive  aiding.  Chapter  8  in  M.D.  McNeese  &  M.A.  Vidulich  (eds.) 
Cognitive  systems  engineering  in  military  aviation  environments:  Avoid  cogminutia 
fragmentosa!  Wright-Patterson  AFB,  OH:  HSIAC,  253-314,  Found  at:  http://iac.dtic.mil 
/hsiac/SOARs_TOCs.htm. 

“The  UK  Ministry  of  Defence  (MOD)  in  conjimction  with  the  Defence  Evaluation 
Research  Agency  (DERA)  have  established  a  program  of  applied  research  concerned 
with  the  development  of  cockpit  adaptive  automation  and  decision  aiding  for  military 
fast-jet  pilots.  The  operational  requirement  for  this  cognitive  cockpit  project  arises  from 
the  possibility  of  a  highly  automated  future  offensive  air  system,  involving  a  mix  of 
manned  and  uninhabited  air  vehicles.  In  complex,  rapidly  changing  military  environ¬ 
ments,  increased  dependencies  on  automation  present  significant  challenges  to  maintain¬ 
ing  effective  human  cognitive  involvement  in  systems  functioning.  A  human-centered 
approach  to  system  design  is  needed  that  is  based  on  human  cognitive  requirements  for 
the  control  of  system  functional  purpose,  decision-making  usability,  and  effectiveness  in 
context  of  use.  Technology  is  needed  to  assist  rather  than  replace  the  future  aircrew  in 
cognitive  work  with  systems  involving  high  levels  of  task  automation.  Support  will  be 
needed  that  is  adaptive  and  context-sensitive,  to  be  responsive  to  changing  mission 
requirements,  in  particular  for  in-flight  situation  assessment  and  mission  replanning,  in 
ofiier  words,  decision  support  to  provide  the  right  information,  in  the  right  way,  and  at  the 
right  time.  Technology  needs  to  consider  the  aircrew’s  physiological  and  behavioral 
state,  adaptively  responding  to  an  individual’s  indications  of  overload,  distraction,  and 
incapacitation.  This  chapter  describes  a  program  of  research  in  cognitive  systems  engi¬ 
neering  that  seeks  to  couple  pilot  functional  state  assessment,  knowledge-based  systems 
for  situation  assessment  and  decision  support,  with  concepts  and  technologies  for  adap¬ 
tive  automation  and  cockpit  adaptive  interfaces.  The  intention  is  to  provide  a  scientific 
quantitative  assessment  of  a  broad  range  of  options  for  intelligent  pilot-aiding.  This  is  to 
be  b^ed  on  sound  cognitive  systems  engmerang  principles  for  system  cognitive  control, 
which  keeps  the  pilot  in  control  of  the  system,  rather  than  the  system  controlling  the 
pilot.” 

Tokamani,  O.,  Kaida,  K.,  Ashida,  H.,  Mizumoto,  C.,  &  Tatsuno,  J.  (1998).  Visual  influence 
on  the  magnitude  of  somatogravic  illusion  evoked  on  advanced  spatial  disorientation 
demonstrator.  Aviation,  Space,  and  Environmental  Medicine, 
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The  somatogravic  illusion  (SGI)  is  a  kind  of  spatial  disorientation  caused  by  a  linear  sus¬ 
tained  acceleration.  Pilots  believe  that  visual  cues,  such  as  a  visible  horizon  or  texture 
flow,  reduce  this  illusion.  This  study  was  performed  to  evaluate  the  influence  of  visual 
stimuli  on  the  SGI  using  the  Advanced  Spatial  Disorientation  Demonstrator  (ASDD). 
There  were  eight  healthy  males  who  were  exposed  to  a  0.58  g  x  axis  linear  acceleration 
on  the  ASDD,  where  the  direction  of  the  resultant  gravito-inertial  force  was  equivalent  to 
30  degrees  pitch-up.  One  of  the  following  visual  stimuli  was  presented  during  each 
acceleration:  BLANK  (no  visual  cues);  HORIZON  (a  visible  horizon  without  motion); 
and  TEXTURE  (vertical  lines  moving  toward  the  subject  evoking  vection).  The  subjec¬ 
tive  magnitude  of  the  SGI  in  ordinal  scale  was  observed;  and  in  interval  scale,  the  devia¬ 
tion  of  the  moving  point  kept  at  the  subjective  horizon  was  observed.  The  differences 
among  visual  stimuli  were  analyzed.  The  subjective  magnitude  of  the  SGI  (p  <  0.01)  and 
the  deviation  of  the  moving  point  (p  <  0.05)  were  significantly  smaller  in  HORIZON  than 
in  BLANK  and  TEXTURE.  No  difference  was  demonstrated  between  BLANK  and 
TEXTURE.  The  linear  vection  produced  by  the  TEXTURE  stimulus  did  not  affect  the 
SGI.  The  data  indicated  that  the  presence  of  a  visible  horizon  reduced  the  magnitude  of 
the  SGI.  On  the  other  hand,  the  presence  of  a  vection  stimulus  did  not  influence  the 
magnitude  of  the  SGI. 
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TX:  Spatial  Disorientation  Symposium.  Found  at:  http://www.spatiald.wpafb.af.miFMishap% 
20Studies/Veronnenau.pdf 

Presents  NTSB  and  FAA  statistics  for  civilian  SD  mishaps  irom  lysz-iyyi.  Acciaents 
typically  involve  non-instrument-rated  pilots  attempting  to  complete  VFR  flights  in  IMC. 
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Welch,  R.B.,  &  Warren,  D.H.  (1986).  Intersensory  interactions.  In  K.R.  Boff,  L.  Kaufinan,  & 
J.P.  Thomas  (Eds.),  Handbook  of  Perception  and  Human  Performance  (V olume  I),  pp,25.1- 
25.36.  New  York:  John  Wiley  &  Sons. 

“. .  .[T]he  procedure  of  examining  each  of  the  senses  as  if  it  were  independent  of  the  oth- 
ere  can  lead,  at  best,  to  only  partial  understanding  of  everyday  perceptual  experience  ” 

.  .[F]or  human  beings  vision  is  more  precise  and  accurate  than  any  other  modality  for 
the  perception  of  spatial  events,  whereas  audition  is  best  suited  for  the  perception  of  tem¬ 
poral  events.”  “The  preponderance  of  evidence  indicates  that  when  the  accessory  stimu¬ 
lus  is  of  low-to-moderate  intensity  the  effect  on  the  primary  stimulus  is  one  of  facilita¬ 
tion,  whereas  highly  intense  accessory  stimuli  are  inhibitory.”  Another  important  vari¬ 
able  is  the  timing  and  duration  of  each  stimulus.  Spatial  events  and  “the[ir]  order  of 
decreasing  relative  acuity. .  .[are]  vision,  touch/kinesthesis,  and  audition.”  Touch  and 
pressure  have  strong  effect  on  orientation.  “. . .  [T]he  perception  of  visual  direction 
requires  the  presence  of  information  regarding  felt  ocular  and  head  position  and  that  if 
this  information  is  incorrect,  the  perception  is  also  incorrect.”  “According  to  the 
directed-attention  hypothesis,  the  degree  to  which  two  sensory  modalities  bias  one 

anodier  is  determined  by  tiie  observer’s  distribution  of  attention  to  each  of  them.  Thus  if 
a  great  deal  of  attention  is  paid  to  one  modality  relative  to  the  other,  the  former  will 
strongly  bias  the  latter  but  will  not  be  biased  very  much  itself.” 

Wenzel,  E.M.  (1993).  Localization  in  virtual  acoustic  displays.  Presence,  i(l),  80-107. 

“This  paper  discusses  the  development  of  a  particular  spatial  display  medium,  tiie  virtual 
acoustic  display.  Although  the  technology  can  stand  alone,  it  is  envisioned  ultimately  to 
be  a  component  of  a  larger  multisensory  environment  and  will  no  doubt  find  its  greatest 
utility  in  that  context.  A  general  philosophy  of  the  project  has  been  that  the  development 
of  ^vanced  computer  interfaces  should  be  driven  first  by  an  imderstandmg  ot  human 
perceptual  requirements,  and  secondarily  by  technological  capabilities  or  constraints.  In 
expanding  on  this  view,  the  paper  addresses  why  virtual  acoustic  displays  are  useful. 
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characterizes  the  abilities  of  such  displays,  reviews  some  recent  approaches  into'  their 
implementation  and  application,  describes  the  research  project  at  NASA  Ames  in  some 
detail,  and  finally  outlines  some  critical  research  issues  for  the  future.” 

Wickens,  C.D.  (1984).  Processing  resources  in  attention.  In  R.  Parasuraman  &  D.R.  Davies 
Varieties  of  attention.  New  York;  Academic  Press,  63-258. 
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Kaufinan,  &  J.P.  Thomas  (Eds.),  Handbook  of  Perception  and  Human  Performance  (Volume  II), 
pages  39.1-39.60.  New  York:  John  Wiley  &  Sons. 

Motion  effects  on  the  vestibular  system,  display  guidance,  and  auditory  redundancy  to 
visual  displays.  A  table  summarizes  the  roles  of  the  utricle  and  semicircular  canals  in 
various  flight  situations,  and  the  resulting  illusions.  Vision  fixation  is  also  discussed: 
“the  dwell  of 400-600  msec  is  the  minimum  time  necessary  to  acquire  foveal  information 
appropriate  for  control  action.”  “. . . [VJery  rapid  dwells  of  125-200  msec. .  .appear  to  he 
‘global  checks’  to  affirm  that  the  indicator  is  in  the  right  general  location,  rather  than 
dwells  defined  to  extract  quantitative  information.”  Wickens  later  asserts  that  integrated 
displays  are  naturally  the  best  for  scanning  and  control,  and  that  “a  single  error  indicator 
[that]  can  make  excursions  in  more  than  one  orthogonal  axis  should  be  of  considerable 
benefit  to  tracking  performance.”  This  rationale  supports  our  SD  icon  design  (Figure  7). 
Later  in  the  chapter,  in  a  section  on  auditory  displays,  Wickens  states,  “. .  .auditory  dis¬ 
plays  of  the  analog  information  used  in  tracking  can  never  be  expected  to  replace  visual 
displays.  However,  their  value  as  a  redundant  source  of  information  can  be  realized  in 
situations  when  visual  work  load  is  already  high.” 

Wickens,  C.D.  (1995).  Designing  for  stress.  In  J.  Driskell  &  E.  Salas  (Eds.),  Stress  and  Human 
Performance.  Mahwah,  NJ:  Erlbaum. 

Wickens,  C.D.  (2002).  Situation  awareness  and  workload  in  aviation.  Current  Directions  in 
Psychological  Science,  11(4),  12S-133. 

“Designers  need  to  imderstand  these  three  general  issues  [fi-ame  of  reference,  degree  of 
integration,  prediction]  and  integrate  the  principles  of  visual  attention,  spatial  cognition, 
and  manual  control  to  craft  the  configuration  of  aircraft  displays  that  will  support  all  of 
the  needed  tasks,  without  overloading  attentional  capabilities.”  Auditory  tasks  tend  to  be 
more  interrupting  and  less  intermptible  than  tasks  even  with  a  higher  priority. 

“. .  .[E]frective  models  of  mental  workload  should  be  able  to  predict  the  circumstances  in 
which  the  workload  of  routine  performance  is  raised  to  such  a  level  that  resources  are  not 
available  to  maintain  situation  awareness. . .” 

Wickens,  C.D.  (2003).  Aviation  displays.  In  P.  Tsang  &  M.  Vidulicti  (Lds.j,  fnncipies  ana 
practices  of  aviation  psychology.  Mahwah,  NJ:  Lawrence  Erlbaum,  147-199. 
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Wickens,  C.D.,  &  Hollands,  J.G.  (2000).  Engineering  Psychology  and  Human  Performance. 
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Wickens,  C.D.,  &  Rose,  P.N.  (2001).  Human  factors  handbook  for  displays:  Summary  of 
findings  from  Army  Research  Lab ’s  advanced  displays  and  interactive  displays  federated 
laboratory.  Thousand  Oaks,  CA:  Rockwell  Sciences. 

“This  handbook  addresses  many  relevant  issues  for  enhancing  human  information  proc¬ 
essing.  Guiding  attention  with  cueing,  highlighting,  and  alarms  or  alerts  have  proven 
value.  Flashing  is  especially  compelling.  Target  tunneling  is  one  potential  down-side  to 
focusing  attention  on  one  item  to  the  exclusion  of  others.  Information  rich  displays  have 
clutter  issues,  which  can  be  alleviated  by  filtering  and  decluttering,  both  of  which  have 
drawbacks,  namely  determining  a  priori  what  is  or  is  not  essential  information.  Color 
coding  is  useful;  intensity  coding  is  generally  less  useful.  Spatial  sqjaration  techniques 
seem  promising,  but  additional  research  is  needed.  Choosing  the  type  of  display  (head- 
moimted,  see-through,  or  traditional  CRT  or  LCD),  and  the  frame  of  reference  are 
important  considerations  in  d^ign,  especially  for  orientation  and  navigation  tasks. 
Enhancing  situation  awareness  is  a  goal  of  display  design;  information  integration  aids 
achieving  and  maintaining  SA.  Automated  enhancement  of  infonnation,  b^ed  upon  the 
user’s  inferred  intent,  is  one  solution  to  the  information  overload  problem.” 
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X.  Acronyms  &  Terms 

3D  three-dimensional 

A  attitude 

AFB  air  force  base 

AFRL  Air  Force  Research  Laboratory 

AFS  A  Air  Force  Safety  Agency 

AFSC  Air  Force  Safety  Center 

AGL  (altitude)  above  ground  level 

AI  attitude  indicator 

Alt  altitude 

ANOVA  analysis  of  variance 

AOA  angle  of  attack 

Ap  perceived  attitude 

ASDD  advanced  spatial  disorientation  demonstrator 

At  true  attitude 

attn  attention 

B  bank 

Baro,  baro  barometric  (altitude) 

Bp  perceived  bank 

C#  a  computer  language  (pronounced  “see  sharp”) 

C-17  4-engine  US  AF  cargo  aircraft 

CF  Canadian  forces 

CF-18  Canadian  fighter  attack  jet 

CFIT  controlled  flight  into  terrain 

Class  A  mishap  that  costs  >  $1M,  or  has  loss  of  life,  or  causes  permanent  total  disability, 
or  destroys  an  aircraft 

Class  B  mishap  that  costs  >  $200K  and  <  $1M,  or  permanent  partial  disability,  or 
hospitalization  of  3  or  more  people 

Class  C  mishap  that  costs  >  $20K  and  <  $200K,  or  injury  that  results  in  lost  work  after  the 
day  or  shift  it  occurred,  or  disability  (soiuce:  http://www2.faa.gov/arp/ 
environmental/5054a/MOAFINAL VERSION.doc) 

CRE  control  reversal  error(s) 

CRT  cathode  ray  tube 

DA  difference  in  attitude:  |Ap-At| 

deg  degree(s) 

DERA  Defence  Evaluation  Research  Agency  (UK) 

EEG  electroencephalogram 

F- 1 5  US  AF  two-seat  high  performance  fighter  attack  j  et 

F-22  USAF’s  newest  stealthy  fighter  attack  jet 

FAA  Federal  Aviation  Administration 

FDM  flight  data  monitoring  (European  term  for  FOQA) 

FMA  follow-me  aircraft 

fimptmnal  mjigriptiV.  rpi.Qnnanra 

FOQA  flight  operations  quaUty  assurance  (routine  data  collection  and  analyses) 

FOR  fi’ame  of  reference 

FOV  field  of  view 
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FY 

G 

GAIN 

GCAS 

GD 

G-LOC 

HDD 

hdg 

HECI 

HITL 

HITS 

HMD 

HUD 

Hz 

IAS 

EFR 

ICAO 

IMC 

IRB 

ISD 

LSA 

Lt 

MA&D 

MAAD 

mag 

Micro  Saint 

MFOQA 

MOD 

MRT 

msl 

NTSB 

NVG 

OKCR 

P 

PET 

PI 

PITL 

Pp 

RAlt 

rCBF 

RSD 

SA 

SAWOY 

SBIR 


fiscal  year 

acceleration  due  to  gravity 

global  aviation  information  network 

ground  collision  avoidance  system 

grapefruit  display 

G-induced  loss  of  consciousness 

head-down  display 

heading 

Crew  Systems  Development  Branch  (part  of  HEC,  the  Crew  Sj^tem  Interface 

Division,  which  is  part  of  tiie  Human  Effectiveness  Directorate,  HE) 

human  in  the  loop 

highway  in  the  sky 

hehnet-mounted  display 

head-up  display 

Hertz  (cycles  per  second;  so,  2  Hz  is  2  cycles  per  second) 

indicated  airspeed 

instrument  flight  rules 

Intemational  Civil  Aviation  Organization 

instrument  meteorological  conditions  (poor  visibility) 

institutional  review  board 

incapacitating  SD 

loss  of  SA 

lieutenant 

Micro  Analysis  &  Design 
Micro  Analysis  &  Design 
magnetic 

MA&D’s  discrete  event  simulation  tool 
miUtary  FOQA  (the  military’s  FOQA  program) 

Ministry  of  Defence  (UK) 

(Wickens’)  multiple  resource  theory 
(altitude  above)  mean  sea  level 
National  Transportation  Safety  Board 
night  vision  goggles 
opto-kinetic  cervical  reflex 
pitch 

positron  emission  tomography 
principal  investigator 
pilot  in  the  loop 
perceived  pitch 

radio  or  radar  altitude  (hei^t  above  the  ground) 
regional  cerebral  blood  flow 
recognized  SD 
situation  awareness 

somatic,  auditory,  visual,  v^tibular,  olfactory,  psychomotor  and  cogmtive 

workload  scores 

Small  Business  Innovation  Research 
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SD 

sec 

SGI 

SO 

SVS 

SXGA 

TCAS 

TLS 

TSAS 

Type  1  SD 

Type  2  SD 

Type  3  SD 

UAV 

UH-60 

UK 

US,  U.S. 
USAF 
USD 
V-22 

VACP 

VAP 

Var 

VFR 

VMC 

VOR 

VSI 


spatial  disorientation 
second(s) 

somatogravic  illusion 

spatial  orientation 

synthetic  vision  system 

super  extended  graphics  array 

traffic  alert  and  collision  avoidance  system 

tactor  locator  system 

tactile  situational  awareness  system 

unrecognized  SD 

recognized  SD 

incapacitating  SD 

uninhabited  air  vehicle 

US  Army’s  frontline  utility  helicopter 

United  Kingdom  (England) 

United  States 
United  States  Air  Force 
unrecognized  SD 

tilt  rotor,  vertical/short  takeoff  and  landing,  multi-mission  aircraft  used  primarily 
by  the  US  Navy,  Marine  Corps,  and  Air  Force 

visual,  auditory,  cognitive,  psychomotor  -  resource  components  which  comprise 
Wickens’  multiple  resource  theory 
variance  of  perceived  attitude 
variance 

visual  flight  rules 

visual  meteorological  conditions  (good  visibility) 
vestibulo-ocular  reflex 
vertical  speed  indicated 
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